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THE IMPORTANCE OF ENVIRONMENTAL EDUCATION AT AN EARLY AGE 


Vincentas Lamanauskas 
Vilnius University, Lithuania 


There is no doubt that environmental education at an early age is significant for human development. When 
we refer to an early age, we mean the preschool and the primary stage of education (preschool education institution 
and primary school). It is obvious that the foundations of personality are formed at this stage, primarily a positive 
attitude towards nature and the surrounding world. At this stage, the child begins to distinguish himself from the 
environment, an emotional and value attitude towards the environment, nature, himself, and other people, etc., is 
formed. On the other hand, knowledge corresponding to a certain age of a personality (e.g., ecological), norms and 
rules of interaction with nature are required, on the other hand, it is extremely important to develop empathy for 
nature through various activities (e.g., actively solving certain environmental problems). It can be reasonably stated 
that the formation of values, nature cognition and a sustainable lifestyle are three essential factors determining 
(describing) the importance of environmental education. Thus, environmental education at an early age not only 
helps to develop environmental awareness but also promotes abilities and the attitude, which can determine a 
sustainable and responsible lifestyle in the future. From this point of view, the continuity and sustainability of this 
education is extremely significant. Environmental education in primary school can have a long-term effect on chil- 
dren's moral and worldview behaviour and contribute to sustainable development and global well-being. The period 
of primary education is a suitable and favourable environment for the formation of children’s positive and rational 
attitude towards the environment, promoting environmental awareness and developing a sustainable lifestyle. 

The preschool educational institution is the first link in the continuous environmental education system, so it 
is not by chance that teachers are faced with the task of forming the cultural foundations of preschoolers’ rational 
environmental management. Later, the accumulated experience is transferred to the primary school. Consistency 
and continuity are important here. It is fully understood that environmental education is not a one-time event or 
activity but must be consistently integrated into the teaching process at all levels and stages of the school. Continu- 
ity ensures that important principles of sustainability and environmental habits are formed from early childhood 
and developed throughout formal and non-formal education periods. 

Environmental protection problems are faced all over the world. The quality of the environment has a ten- 
dency to deteriorate, therefore environmental education becomes one of the essential conditions for continued 
existence. In order to improve the situation, it is necessary to raise public awareness and encourage behaviour 
change. It is obvious that environmental education is needed, which would raise the level of people’s awareness, 
encourage them to change their behaviour, accordingly, changes would take place in the field of production and 
industry, consumption habits and the relationship with the environment itself. Environmental education is espe- 
cially important in primary school. In forming children’s environmental awareness, a large responsibility falls on 
the primary school teacher, therefore, his preparation in the field of environmental education must be adequate 
(Lamanauskas & Makarskaité-Petkeviciené, 2023). It is emphasized that the requirements for the environment are 
growing, and it is increasingly necessary to act sustainably, responsibly, and respectfully, not only to protect but 
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also to restore the environment. In the process of environmental education, various didactic tools, approaches, 
and techniques are applied. Environmental education content is created and updated, but not always properly 
and in a timely manner. Researchers emphasize that digital technologies including video conferencing, mobile 
apps, and virtual and augmented reality, can provide new ways to engage children/students in environmental 
maintenance (Buchanan et al., 2019). However, there remains a fundamental question that should be a focus 
of those interested in environmental education. Existing and developing digital technologies can become both 
encouraging and hindering elements in educating young people to think globally and to behave responsibly and 
respectfully in the surrounding environment. 

Speaking about the most important barriers to environmental education, it should be said that they basically 
remain the same. For example, a study conducted in 1988 (Sewing & Ham, 1988) showed that the most important 
barriers were lack of time (both for the lesson and preparation), lack of teaching materials and funding, and lack 
of teaching competence of teachers. However, recent studies also show that teachers lack knowledge and proper 
teaching on environmental topics, which in turn causes difficulties in implementing environmental education 
programmes. For example, a study conducted in Greece showed that teachers were interested in environmental 
issues and mainly used the media to get information about environmental protection, however, there are also 
significant shortcomings in the development of teachers’ abilities, the organization of environmental education 
in preschool and primary education institutions, which negatively affect the implementation of environmental 
programmes in schools (Petkou et al., 2021). Many researchers rightly believe that the solutions to environmental 
problems are influenced by accepted environmental ethics. For example, a study conducted in Poland in 2017 
(analysed the content of textbooks used by primary school science teachers in the fourth grade) showed that the 
content of formal environmental education is dominated by anthropocentric ethics, and biocentric ethics is much 
less common, and only very little attention is paid to holistic ethics and the declaration that man is a part of nature 
(Gola, 2017). A study conducted in Serbia showed similar results. Recent changes in school policies and educational 
plans/programmes confirm that the relevance of environmental education has been recognised, however, there 
are practically no changes in school practices (Stanisi¢ & Maksi¢, 2014). Thus, in the educational programmes of 
many countries, environmental education is not always sufficiently emphasised or may even be practically omitted 
or even ignored. As a result, teachers may not gain sufficient knowledge and confidence in this important area. On 
the other hand, environmental education is constantly changing and developing, new perceptions and didactic 
approaches appear. It is obvious that it is difficult for teachers to keep up with all these new developments and 
incorporate them into their lessons, especially if they do not have the opportunity to learn new things regularly. 
Another well-known truth is that primary school teachers often have multiple tasks, including teaching general 
subjects, socialising with children and more. Environmental education can remain somewhat suppressed (offside) 
due to many other priorities. It is important to emphasize that attention must be paid to the training and support 
of teachers at all levels, to ensure that they are prepared and able to effectively integrate environmental education 
into their lessons. And here, the preparation of students, future preschool and primary education teachers in the 
field of environmental education during their studies at university is no less important. Here are some excerpts 
from respondents’ answers about their vision of implementing environmental education at school (Lamanauskas 
& Makarskaité-Petkeviciené, 2023): 


| will encourage the school community to organise activities for environmental education. In our free time, we will watch 
videos with the students and discuss environmental protection. When these topics are included in world cognition lessons, | 
will devote time to telling as much and as original as possible about environmental protection. | think it is possible to imple- 
ment various projects, for example, try to sort out for a week and reflect daily with the students on how they managed to 
do it. Another example is to ask each student to make a poster encouraging them to protect nature or to create a booklet, 
showing how we can contribute to environmental protection. Also, we can participate in the environmental action “Let's 
do it” It would be nice if mini containers were placed in the classrooms for sorting, | think it is useful to set an example that 
sorting can be done notonly in the classroom but also at home. It is important to develop students’ awareness, and to explain 
that they can also contribute to this (Respondent A, Childhood pedagogy - primary education specialisation, IV course). 


The main vision is that every school not only talks about environmental protection but also uses all environmental protec- 
tion methods — to reduce the amount of printed paper, use renewable energy sources, sort waste, and contribute to other 
environmental actions. After all, words remain only words if we do not see good examples in our environment. (Respondent 
B, Childhood pedagogy - primary education specialisation, IV course). 
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Children need to be taught to contribute to environmental protection. It is not always easy to do this. Therefore, it is im- 
portant to create an understanding that even a small step in contributing to the preservation of the environment is very 
useful. (Respondent C, Childhood pedagogy - preschool education, | course). 


In 2023 an empirical qualitative study was conducted in Lithuania, which aimed to reveal how future primary 
school teachers understand the meaning of environmental education, the topic, and what kind of realisation vision 
they have. It was established that future teachers have a fairly clear position on the issue of environmental educa- 
tion. Environmental education is treated as a significant part of general education, it is important to understand its 
integrative and interdisciplinary essence. Students see different areas of environmental education that should be 
covered in primary school. The topics include such areas as waste issues, environmental pollution, global climate 
change, environmental solutions, and the crisis of values. Judging from the point of view of age groups, primary 
school students (7-11 years old) are usually willing to take responsibility, are empathetic, understand the moral 
imperative, their thinking is strongly dependent on experience, they learn quickly from authority figures, so the 
position of the teacher, his active participation in environmental activities, his environmental values are very impor- 
tant. The vision of future teachers’ implementation of environmental education at school includes both a cognitive 
component (strengthening environmental education through the development of knowledge and understanding, 
formation of value attitudes), and a practical- behavioural component (increasing awareness of waste sorting and 
promoting the use of secondary raw materials) (Lamanauskas & Makarskaité-Petkeviciené, 2023). 

The competence of teachers in the field of environmental education may vary depending on the specific 
country, school (especially in terms of its location), individual efforts of teachers and various other factors. However, 
it is often observed that teachers in many countries lack knowledge and confidence in terms of environmental 
education. Many teachers may have basic knowledge about environmental protection; however, they lack specific 
methods and ways how to transfer this knowledge to students and how to help them form sustainable behavioural 
habits. There is also a lack of information about reliable sources of information (e.g., proponents and opponents of 
global climate change), ideas for activities and planning and implementation of activities in this area. 

It can be said that it is important not only to talk about environmental education but also to study its effec- 
tiveness so that we can improve it taking into consideration the obtained results. The study of the effectiveness 
of environmental education is an important process that helps to evaluate the results of this type of education 
and to make sure that it achieves the intended goals and effects on the knowledge, attitudes, and behaviour of 
students. The results of the research can provide valuable knowledge about how to do better and how to improve 
environmental education. The aspect of optimal balance is important. If environmental education is too theoretical 
and not practical enough, or on the contrary, too practical and not sufficiently based on theoretical foundations, it 
may not be useful for both students and teachers. Therefore, it is necessary to ensure that environmental education 
is consistently integrated into curricula in all disciplines (e.g., using STEM methodology and approaches). This will 
enable students to relate sustainability principles with real areas of knowledge. An extremely rational approach is to 
organise practical classes and activities that would allow students to actively learn about nature, conduct research, 
experiments and create projects related to the environment and its protection and improvement. Recently, it is 
significant to effectively use modern technologies, video materials, interactive teaching tools and internet resources, 
which can increase the involvement of students and form their adequate understanding of environmental protec- 
tion. Based on research data, educational institutions can adjust their programmes and methodology. Therefore, 
by conducting research on environmental education effectiveness, it is possible to obtain valuable information 
about its impact and contribute to the consistent and effective development and implementation of this education. 
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Abstract. STEM education is a crucial 
learning area to understand important 
concepts and gain 21st-century skills in 

different disciplines, explore scientific 
knowledge, and solve problems in 

daily life. However, research in STEM 
education is scarce on extended reality 
technologies. This research examined 

the effects of extended reality integrated 
STEM on students’ learning response 

and performance. A quasi-experimental 
research design was used to find answers 
to the research questions. The same lecture 
taught the experimental and control 
groups but preceded STEM experimental 
activities at a different time and with 
distinct instructional strategies and 
teaching aids. The participants were 
students in two classes at a university 

in Taiwan. The research results showed 
that students in the experimental group 
applying extended reality-integrated 

STEM had better learning responses 

than students in the contro! group using 
teaching materials. The results also 
showed that students in the experimental 
group learning with an extended reality 
system could apply the STEM curriculum 
knowledge acquired in classes and 

reflect the learning outcomes on the test 
assessment for better learning performance 
than students in the control group. In 
addition, the results revealed that good 
learning responses could enhance students’ 
higher learning performance. In light of the 
results, educational implications are made. 
Keywords: extended reality, STEM 
education, learning response, learning 
performance, experimental research 
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THE EFFECTS OF EXTENDED 
REALITY TECHNOLOGIES 

IN STEM EDUCATION ON 
STUDENTS’ LEARNING 
RESPONSE AND PERFORMANCE 


Yu-Chen Chiang, 
Shao-Chieh Liu 


Introduction 


The 21st century is a technology-driven era, and the information network 
of the global village makes science and technology education an important 
area of learning for students. The goals of the curriculum include fostering the 
interest and passion for science exploration and the habit of active learning, 
learning the exploration and basic competence of science and technology, 
applying what is learned to present and future life, cultivating the attitude of 
protecting the environment, valuing resources and respecting life, cultivat- 
ing the ability to communicate and express oneself with people, teamwork 
and harmony, cultivating independent thinking and problem-solving skills 
and stimulating creative potential, and awareness and exploration of the 
interaction between people and technology. Policy and reform documents 
in STEM education emphasize that science education not only focuses on 
the teaching of scientific knowledge but regards the cultivation of people’s 
basic scientific ability as an important task, and expects people to be able 
to understand the contribution and limitations of science and technology 
to human beings, recognize the crucial concept and principles of science 
and technology, explore the commonalities and ambiguities, and apply 
scientific knowledge and scientific thinking to deal with personal life, solve 
problems, and even develop the potential to create new products for the 
benefit of people. 

The new developments and advances have led to using artificial intel- 
ligence technologies in education. Until the advent of new technologies, 
lectures had to follow the schedule to arrange and design student teaching 
concepts in the form of narratives (Yannier et al., 2020). However, such a 
teaching method was a one-sided teaching mode that could easily lead to 
students being unwilling to interact with new knowledge and seeming to 
have a negative learning attitude toward expertise. STEM, the abbreviation 
for science, technology, engineering, and mathematics, is promoted inter- 
nationally mainly to help people grasp and present multidimensional skills 
for real-world problems (Lee & Kim, 2021). Meanwhile, STEM encourages 
students to engage in hands-on learning to improve their understanding of 
math and science. It also encourages students to pursue engineering and 
science careers to maintain national industrial competitiveness worldwide 
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(Andrews et al., 2021). Researchers have integrated STEM into artificial intelligence courses. According to the 
research results, STEM-based artificial intelligence courses have promoted and improved students’ application of 
artificial intelligence (Lin et al., 2021). This study applied extended reality in teaching to help users learn through 
practice, understand the teaching content more precisely and deeply, and effectively promote achievement, the 
motivation to accept new knowledge, and the ability to acquire knowledge. At the same time, it can improve the 
insufficient interaction in the previous learning process. Therefore, this study explored the integration of extended 
reality into STEM regarding learning responses and performance, expecting that applying extended reality to the 
classroom experience and learning helps students achieve better learning performance. 


Extended Reality and STEM Education 


Extended reality (XR) is a general and integrated concept covering characteristics of virtual, augmented, and 
mixed reality (Altmeyer et al., 2020). The characteristics of virtual reality are to apply 3D virtual space to replace 
the real environment and provide users with an immersive experience and realistic effect (Karampelas, 2021). The 
characteristics of augmented reality are to augment virtual objects in real scenes for the users’ interaction (Altmeyer 
et al., 2020). On the other hand, the characteristics of mixed reality are to stack virtual objects in the current world 
and emphasize crossing applications between virtual reality and augmented reality (Lee, 2021). XR could be defined 
as applying all real and virtual technologies to the interaction through the man-machine interface or appropriate 
hardware system equipment; meanwhile, there is a cross-relationship between the virtual and real environment 
in extended reality (Huang et al., 2020). 

STEM integrates areas of science, technology, engineering, and mathematics, emphasizes the implementation 
of interdisciplinary concept teaching among various subjects, and allows students to master the curriculum concept 
for applications (Lee & Kim, 2021). Meanwhile, STEM encourages students hands-on to learn through the above 
routes (Lin et al., 2021). Benyeogor et al. (2021) emphasized that STEM should be the model integrating disciplin- 
ary theories with hands-on for students to comprehend and solve problems so that students could present the 
practical application and development skills for STEM teaching. Through hands-on, students comprehended and 
constructed learning impressions in the implementation process to deepen their cognition and avoid being unable 
to acquire and apply the technology by simply reciting theories from books. From the experiment of STEM robot 
coding, Xu and Wang (2022) concluded that teaching with different types of STEM activities would result in junior 
and senior high school students maintaining a positive tendency to STEM. Besides, students with highly positive 
STEM tendencies might be more interested in STEM-related jobs in the future. Jiang et al. (2022) mentioned that, 
in modern society, citizens were gradually requested to present professions integrating knowledge and technol- 
ogy into STEM to solve problems in reality; therefore, students should be reinforced with the ability to integrate 
interdisciplinary concepts and skills. 

Extended reality is currently broadly applied to subject teaching as the visual medium to provide a high- 
interaction virtual learning environment for learners and offer users an immersive experience in the teaching process 
to vastly reduce the probability of being dull in classes (Hu-Au & Okita, 2021). For example, Benyeogor et al. (2021) 
stated that the existence and emergence of extended reality resulted in new definitions of teaching media and 
teaching aids. Many researchers studied the phenomena and influenced learners in immersive simulated learn- 
ing environments. When the simulated world could accurately realize rendering technology was created and the 
cognition of compound illusion was output, learning response would be guided in the highly changeable virtual 
teaching world. Di Natale et al. (2020) conducted a systematic review of empirical research aiming at extended 
reality immersive technology. They revealed that extended reality immersive technology could support STEM 
experiments to induce further students’ interest in STEM material or teaching aid systems. The core advantage lies 
in users acquiring real-time experience that could not be reached in the real world. A unique experiential learning 
response enhanced students’ enthusiasm and participation (Okonkwo & Ade-lIbijola, 2021). 

Gao et al. (2020) indicated that group learning with extended reality integrated STEM outperformed members 
in the other two groups on learning memory, positive emotion, and report participation. It was considered in the 
research that the application of extended reality integrated STEM could present better learning performance than 
conventional teaching and digital video teaching. Liu et al. (2020) preceded a quasi-experiment in which students 
in the experimental group applied extended reality to the STEM curriculum while students in the control group 
learned with conventional materials. Their results revealed that students in the experimental group outperformed 
the control group in learning performance and participation. In another research, Xu and Wang (2022) proposed 
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the training environment of an immersive extended reality engineering laboratory to provide students with expe- 
rience in a natural laboratory. According to them, the advantage of an immersive extended reality learning envi- 
ronment was providing a simple but highly secure experiment course, which could present identical engineering 
experiment content as in reality, for restoring the experience and interaction process. The research data revealed 
the positive learning effect, achievement, and specific practice experience of STEM from the immersive extended 
reality system (Theodoropoulou et al., 2020). 

Guo et al. (2021) developed an extended reality mobile device for the study. The research showed that ex- 
tended reality could effectively support science learning response and cultivate scientific inference skills in STEM. 
The study also revealed that extended reality could change abstract science concepts in STEM into tangible ones 
to appear in fascinating learning situations and overturn the subject education in STEM. Jiang et al. (2022) noticed 
that integrating emerging technology and visualization tools into STEM teaching aids could effectively induce stu- 
dents’ participation in STEM classes, enhance the willingness to contact STEM, induce a better learning response, 
and promote learning performance in STEM. Promoting and cultivating students with the above features could 
have them present STEM skills required in the new century. Klingenberg et al. (2020) proposed that STEM-based 
digital learning websites displaying dialogic teaching and providing students with adaptive teaching modes could 
enhance students’ self-directed learning experience and vastly improve students’ learning response and learning 
performance in STEM (Petrov & Atanasova, 2020). 


Extended Reality Implementations in Science Education 


XR encompasses various technologies through virtualization and focuses on developing augmented reality 
(AR) by providing virtual information about the real world and virtual reality (VR) by simulating an environment 
that replaces reality. Since the use and implementation of XR technology in education are relatively new and is one 
of the research topics emerging in the science education literature, this section presents the results of previous 
studies focusing on AR and VR in STEM education at the undergraduate level. 

For example, Liou et al. (2017) examined the efficacy of 2D images derived from VR and AR systems in an 
inquiry-based astronomy course. According to their findings, students in the AR group performed significantly 
better in terms of learning performance and task performance than those in the VR group. Consequently, these 
results suggested that AR technology assisted students in their lunar phase course. “Ease of use” and “intention” of 
acceptance showed no significant differences between the two groups, whereas “usefulness” and “attitude” revealed 
significant differences. Learners can easily integrate virtual objects and real environments with the AR system's 
features to reduce cognitive load and enhance their education. lbanez et al. (2017) examined the effects of using 
an augmented reality-based simulator in a conventional classroom setting. The participants were 112 physics un- 
dergraduates enrolled in a Spanish college. Students could understand how magnetic fields interact with charged 
particles by viewing their three-dimensional trajectories. Their findings demonstrated that augmented reality 
technology increased student motivation. Artun et al. (2020) conducted a survey to explore the impact of virtual 
reality-enhanced laboratory activities on the science process skills of undergraduate science students who want 
to become lecturers. The study used the Google Expeditions app’s VR environments and smartphone-compatible 
headsets to teach physics, chemistry, and biology concepts. The findings revealed a significant increase in test 
scores of the experimental group compared to the control group. Lee and Shea (2020) aimed to identify disparities 
between two pre-service elementary school lectures in a virtual reality learning platform. The participants were 
asked to assess, design, and appraise virtual reality applications (VR) in a three-stage process. The findings revealed 
that elementary school lectures exhibited increased confidence in teaching science through computer-based 
technologies after the intervention. Additionally, the PSETs responded positively towards their VR experiences 
and regarded VR applications as supplementary teaching tools. The majority of PSET teams were able to produce 
creative VR learning materials, which boosted their self-efficacy. 

In a recent study, Lin et al. (2021) studied how non-engineering students performed in various aspects of 
artificial intelligence (Al). They found that students who had a good understanding of Al also showed an awareness 
of Al ethical concerns. Additionally, the researchers discovered that a STEM-based Al curriculum could enhance 
awareness of Al ethical issues in students with low Al knowledge levels. They recommended that lectures teaching 
general education courses incorporate their method to teach Al knowledge. Another study by Zatarain-Cabada et 
al. (2023) explored how web-based extended reality technologies could be used as a learning tool to teach phys- 
ics. Students were provided didactic material on kinematic and dynamic concepts using an experimental process 


570 


ASS https://doi.org/10.33225/jbse/23.22.568 


Journal of Baltic Science Education, Vol. 22, No. 4, 2023 


= i THE EFFECTS OF EXTENDED REALITY TECHNOLOGIES IN STEM EDUCATION ON STUDENTS’ 
ISSN 1648 3898 Dope LEARNING RESPONSE AND PERFORMANCE 
ISSN 2538-7138 /Online/ (PP.568-578) 


incorporating extended reality. The study found that both virtual and extended reality technologies significantly 
impacted motivation. 


Aim 


While studies in the literature have focused on the effectiveness of augmented and virtual reality on student 
performance; no study has examined student learning responses and performance at the undergraduate level. 
Only a few studies (Theodoropoulou et al., 2020; Zatarain-Cabada et al., 2023) have been conducted on extended 
reality in STEM education in the current literature. However, none of these studies have examined the effects of 
STEM education’s extended reality on participants’ learning responses and performance. 

Therefore, this research determined the effects of extended reality-integrated STEM education on student 
learning responses and performance. The main contribution of this study is to present the design and implementa- 
tion of extended reality-integrated STEM education. The research questions of this study were as follows: 

1. What are the effects of extended reality integrated STEM on students’ learning response and learning 
performance? 
2. What are the effects of learning response on learning performance? 


Research Methodology 
Research Design 


This study employed an experimental research design to examine the effects of extended reality-integrated 
STEM education on student learning responses and performance. A control and experimental group design was 
used to answer the research questions and evaluate the effects of extended reality education on student learning 
responses and performance. In experimental studies, researchers collect data to ensure that the participants receive 
the benefits of treatments. This kind of study may involve administering a specific treatment to an experimental 
group so that the participants eventually receive the beneficial treatment (Creswell, 2009). The research was con- 
ducted between September and December months in 2022. 


Participants 


The study involved 102 undergraduate engineering students from two classes at a public research university 
in Taiwan. The participants were selected through a convenient sampling method and comprised 75 males and 27 
females, with an average age of 20.6. The participants were first-year students from an engineering college. The par- 
ticipants indicated that they used computers and smartphones before participating in this research. The participants 
approved the participation in this research. All of them participated voluntarily. They signed a consent form voluntarily 
to participate in the present research. All participants had basic skills in computers. Two lecturers taught the class. 


Procedures 


In a quasi-experimental design, the participants were split into two groups. The experimental group con- 
sisted of 51 students, while the control group had 51 students. The same lecturer taught both groups, but the 
STEM experimental activity was introduced at different times and with different teaching methods and aids. In 
the experimental process, students in the experimental group were taught the instructional strategy of extended 
reality integrated STEM to the experimental activity so that students could be engaged in the extended reality 
learning environment to comprehend STEM courses. On the other hand, students in the control group studied 
with computers, course materials, and teaching aids, e.g., textbook briefings and digital teaching films. Students 
in both groups had to follow the learned STEM content to achieve the learning tasks. 

In the first week, before the teaching activities in the formal curriculum, the lecturer introduced the process 
of the 18-week experimental activity and the learning tasks for students to understand the course practice and 
explained the learning objectives of STEM teaching based on extended reality. After this introduction, the lecture 
administered the pre-tests to the experimental and control groups to determine the differences in prior knowledge 
of STEM between the two groups. 
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The STEM teaching in this study was practiced three hours per week during weeks 2-17. The lecturer taught the 
experimental group with extended reality integrated STEM for instructional activities. The lecturer explained using 
the extended reality system and the corresponding learning units and objectives. The lecturer used the extended 
reality system during the experimental activities STEM. With the real-time evaluation, the lecturer could provide 
feedback to the participants on the STEM activities in the study to create a good learning environment so that the 
students could understand the error and clarify the correct concept for inference and evolution. 

In addition, the effects of extrinsic factors in this study were kept as low as possible, and the experimental 
time was limited. In addition, the smooth process before the formal experiment was confirmed to ensure the 
smooth process of the STEM experiment. For the control group, the lecturer used computers, course materials, 
and teaching aids, including textbook instruction and digital instructional films, and applied didactic instruction 
to STEM experimental activity in the classroom. Students in both groups who had questions in class could actively 
ask the lecturer. Meanwhile, all instruction and experimentation were conducted in the allotted time to minimize 
the impact of extrinsic factors and ensure the accuracy of experimental data. 

The teaching materials in this study aimed for students to learn basic knowledge of artificial intelligence while 
cultivating Al application skills. The design of the activities and the flow of the three Al learning units, introduc- 
tion to features, unsupervised learning, and supervised learning, were developed in this study. Students in the 
experimental group applied extended reality Al teaching systems to learn and complete all Al learning systems. 
The students in the control group completed the experimental courses using computers and teaching materials. 
Meanwhile, students in both groups were assessed for their learning effectiveness and the difference in learning 
perception after completing the learning tasks. The Al materials for augmented reality and the tests for the students 
in the experimental group were developed following the interdisciplinary concepts of STEM. 

Lecturers could comprehensively integrate science, technology, engineering, and mathematics concepts 
into instruction and application. The STEM interdisciplinary teaching concepts used in the teaching materials 
are (1) Science: the ideas of characteristics and labels were integrated into fruit science in the “Introduction to 
Characteristics” unit of the Al teaching materials so that students can understand the characteristics (features) and 
differences (labels) of different fruits. (2) Technology: the “Supervised Learning” unit of the Al teaching materials 
enabled students to apply Al concepts well and combine technology applications to solve the set tasks. The com- 
prehensive concept of supervised learning was integrated into extended reality technology so that students could 
collect features by scanning feature cards to achieve the application of supervised learning and complete the Al 
supervised learning unit and technology knowledge teaching activity. (3) Engineering: the unsupervised learning 
unit was designed for students to apply the concept of engineering assembly to collect the features of different 
fruit labels to complete the unsupervised learning units. (4) Mathematics: Some questions combined parts of the 
calculation ideas with Al concepts in the test paper. For example, the Al concept was integrated with mathematical 
concepts so that students could become familiar with the Al knowledge and use the calculation logic to calculate 
the number of features of two different objects with the same labels, as well as review the core concept of Al and 
promote students’ computational logical thinking. 

In the 18th week of the experiment, the lecturer conducted a post-test for the experimental and control groups 
and STEM learning response questionnaires to discuss the difference between extended reality and STEM in learning 
performance and learning response. Students in both groups completed post-test questionnaires after completing 
the learning activity. It was observed that approximately 20 minutes were adequate to complete the questionnaires 
and survey. The experimental process was conducted between September 15 and December 15, 2022. 


Data Collection Tools 
The experiment tools applied to this study contain the learning response and performance questionnaires. 
Learning Response Questionnaire 


The learning response questionnaire for this study refers to the questionnaire designed by Chang and Chen 
(2020). Dimensions of “organization,”“clarification,"“repetition;“supervision,“concentration,’“effort,’ and “teaching 
aid system’ are discussed in the questionnaire. The learning response questionnaire was used to review and discuss 
the difference in the learning response to STEM curriculum between the experimental and control groups and 


further discuss factors in the result. This study used a course perception questionnaire developed by Chang and 
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Chen (2020). The questionnaire included four dimensions: Learning Materials, Lecturer’s Instruction, Administrative 
Service, and Learning Activity. The questionnaire contained 25 items on learning materials, lecturer's instruction, 
course schedule, learning difficulties, and four questions on administrative service. It was a 5-point Likert scale 
ranging from 5 to 1 in the order of strongly agree, agree, usually agree, disagree, and strongly disagree. The Cron- 
bach’s a-reliability of the six dimensions was higher than .8, indicating that each dimension had adequate reliability. 


Learning Performance Questionnaire 


The learning performance questionnaire evaluated students’ performance during the activities. The question- 
naire was developed by Lin et al. (2021) with a single dimension. Seven question items were used to determine 
learners’ learning performance related to extended reality after class. These items aimed to give the researchers 
better indications of participants’ learning performance concerning extended reality design and learning activi- 
ties. The questionnaire was also designed as a 5-point Likert scale, with 1 representing “strongly disagree” and five 
representing “strongly agree’ The items helped participants describe the current state of their learning performance. 


Data Analysis 


Analysis of variance was utilized in this study to determine the differences between the participants’ learn- 
ing responses and learning performance in the two groups. In addition, regression analysis was further applied 
to understand the relationship between learning response and performance. The pre-test of both groups was 
analyzed with an independent sample t-test, and the result reveals that both groups do not achieve significance 
in the pre-test (p = .751). During the data analysis, researchers used the SPSS program. 


Research Results 
An Analysis of Results Between the Control and Experimental Groups on Learning Response 


According to the analysis of variance to find out the difference of extended reality integrated STEM in learn- 
ing response, i.e., analysis and explanation of learning response, extended reality integrated STEM showed higher 
organization (3.87 > 3.14), clarification (3.98 > 3.46), repetition (4.05 > 3.51), supervision (4.12 > 3.49), concentration 
(4.18 > 3.62), effort (4.27 > 3.71), and teaching aid system (4.38 > 3.69) than the control group, as shown in Table 1. 
This finding is by an analysis of variance that explored the differences between extended reality-integrated STEM 
and the control group regarding learning response. As a result of this finding, H1 is supported. 


Table 1 
Difference Analysis of Extended Reality Integrated STEM in Learning Response 


Variable F p Scheffe post hoc 
Organization 11.387 < 001 Extended reality integrated STEM (3.87) > control group (3.14) 
Clarification 21.524 < 001 Extended reality integrated STEM (3.98) > control group (3.46) 
Repetition 16.375 < .001 Extended reality integrated STEM (4.05) > control group (3.51) 
Supervision 19.162 < .001 Extended reality integrated STEM (4.12) > control group (3.49) 
Concentration 23.516 < .001 Extended reality integrated STEM (4.18) > control group (3.62) 
Effort 25.483 < .001 Extended reality integrated STEM (4.27) > control group (3.71) 
Teaching aid system 31.691 < .001 Extended reality integrated STEM (4.38) > control group (3.69) 
p<.05 


An Analysis of Results between the Control and Experimental Groups on Learning Performance According to the 
analysis of variance to determine the difference between extended reality integrated STEM in learning performance, 
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i.e., analysis and explanation of learning performance, extended reality integrated STEM reveals higher learning 
performance (4.43>3.62) than the control group. This result demonstrates that the analysis and explanation of 
learning performance, extended reality integrated STEM reveals a higher learning performance than the control 
group, which provides support to the hypothesis that H2. 


Table 2 
Difference Analysis of Extended Reality Integrated STEM in Learning Performance 


Variable F p Scheffe post hoc 
Learning performance 42.163 < 001 extended reality integrated STEM (4.43)> control group (3.62) 
p<.05 


Correlation Analysis of Learning Response and Learning Performance 


To test H3, the analysis results, Table 3, reveal significant and positive effects of organization, clarification, 
repetition, supervision, concentration, effort, and teaching aid system (Beta = 0.206*; 0.213**; 0.224**; 0.233**; 
0.239**; 0.245**; 0.261**) in learning response on learning performance that H3 is supported. 


Table 3 
Analysis of Learning Response to Learning Performance 


Dependent variable — 
Learning performance 


Independent variable | 


Learning response Beta p 
Organization 0.206* 016 
Clarification 0.213** .0001 
Repetition 0.224** .0001 
Supervision 0.233** .0001 
Concentration 0.239** .0001 
Effort 0.245** .0001 
Teaching aid system 0.261** .0001 
F 67.839 
R2 0.472 
adjusted R2 0.463 


*p<.05.** p<.01. 


According to this result, the results presented in Table 3 reveal significant and favorable effects of organiza- 
tion, clarification, repetition, supervision, concentration, effort, and teaching aid system. This result suggests that 
Hypothesis 3 is supported. 


Discussion 


This research was to determine the effects of extended reality-integrated STEM education on student learning 
responses and performance. The main contribution of this study was to present the design and implementation 
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of extended reality-integrated STEM education. Given that XR technologies used in this research are a new way 
of teaching STEM subjects to students, this way of teaching provided a mixture of augmented, virtual, and mixed 
reality as a novel approach to promote learning in STEM fields. From this point of view, XR represents a new ap- 
proach to STEM education by enabling students to learn and understand subjects related to STEM and abstract 
concepts through problem-solving using the knowledge base in STEM fields. The XR was designed as a teaching 
method that provides interactive environments to facilitate meaningful learning. In this study, students used XR 
technologies to learn STEM-related topics. 

Research results showed significant effects of extended reality-integrated STEM teaching on learning response. 
According to the results, extended reality integrated STEM teaching had statistically better results on learning 
performance than those in the control group. This result is because extended reality technology could support 
many teaching experiments to stimulate students’ interest in teaching materials or teaching support systems. The 
possible reason for this result could be that the main advantage of extended reality-integrated STEM teaching is 
the ability of users to have real-time experiences that are not possible in the real world, create unique learning 
experiences, and increase student enthusiasm and engagement. This finding aligns with the study conducted by Di 
Natale et al. (2020), which explored the impact of AR or VR technology on learning outcomes during STEM activities. 

The results showed that the XR technologies used in this study helped to improve students’ learning per- 
ception and performance because it is important to point out that these technologies motivate students and 
make them maintain attention to the subject matter (Zatarain-Cabada et al., 2023). As indicated by Zatarain- 
Cabada et al. (2023), the significance of developing specific educational materials, including the arrangement 
of the learning space, the execution of effective instructional exercises, and the precision and excellence of 
instructional materials are important for increasing students’ perceptions. These factors significantly impact 
the level of learning and performance achieved by learners. The research results showed remarkable effects of 
extended reality STEM on learning performance, consistent with the study of Xu and Wang (2022). The training 
environment for an immersive extended reality engineering lab provides students with experiences like those 
in areal lab. The advantage of an immersive extended reality learning environment is that it provides a simple 
and highly secure laboratory course, presents identical engineering experiment content in reality, and recreates 
the realization and interaction process. The research data and results showed that an immersive extended reality 
system could produce a positive learning effect, achievement, and specific practical experiences. The research 
results showed significant effects of learning response on learning performance, consistent with the research 
of Jiang et al. (2022). Integrating new technologies and visualization tools into artificial intelligence teaching 
materials could effectively promote students’ participation in artificial intelligence classes, increase students’ 
willingness to engage with artificial intelligence, stimulate students’ feedback for better learning responses, and 
promote students’ learning performance in artificial intelligence. In a similar research study, Zatarain-Cabada 
et al. (2023) evaluated students’ perceptions of web-based XR technologies for teaching by using a learning 
application that combines virtual reality and augmented reality environments to provide students with the 
necessary didactic materials to learn physics. According to their findings, both virtual and augmented reality 
significantly affected motivation. However, the impact of activities in AR mode on student motivation to learn 
was greater than in VR mode. With this respect, the results of Zatarain-Cabada et al. (2023) are similar to those 
of the present study. The earlier studies and the present study suggested that well-designed learning tools, 
including extended reality, can greatly impact a learner's performance. Factors such as the quality, accessibility, 
and clarity of instructional materials are crucial in achieving this goal (Zatarain-Cabada et al., 2023). The results 
are also similar to those of Ibanez et al. (2020), who reported significant effects of augmented reality-based 
learning environments using the web-based application on learning outcomes, and the augmented reality 
learning environment was more effective for learning than the web-based learning environment. The results 
obtained are also very similar to the findings of Liou et al. (2017), who revealed that augmented and virtual 
reality had significant effects on learning performance. The results showed that the XR technologies demon- 
strated a positive and significant effect of learning response on learning performance. XR technologies in this 
study created a new and effective way for students to learn in STEM education. It gave them a unique learning 
environment and instructional strategy that improved their performance. Researchers have reported that the 
implementation and use of augmented and virtual reality technologies enabled students to focus on STEM 
disciplines and topics and helped them better understand the concepts presented in STEM activities (Akgayir 
et al., 2016; Ibanez et al., 2016; Liou et al., 2017). 
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Conclusions and Implications 


STEM can be applied to real life with extended reality technologies that push students to learn knowledge 
and skills. STEM presents interdisciplinary interests, experiences, situations, collaboration, design, implementa- 
tion, and technology. Regarding student learning, it is important to consider their subject matter knowledge and 
life experience and focus on their needs. In this study, researchers applied an extended reality-integrated STEM 
experimental activity to the students in the experimental group. At the same time, for the students in the control 
group, the teaching was carried out with the help of computers, course materials, and teaching aids. The research 
results showed that the experimental group students who applied the extended reality-integrated STEM achieved 
better learning results than those in the control group who used teaching materials. The post-test data analysis 
showed that the experimental group students outperformed the control group many times. The students in the 
experimental group outperformed the learning performance of the control group STEM. From the data analysis, it 
can be inferred that the students in the experimental group could apply the knowledge from the STEM curriculum 
and reflect the learning outcomes, so they showed better learning performance than the students in the control 
group. The results showed that extended reality technologies could enhance participants’ learning responses and 
performance. 


Recommendations 


It is suggested in the study that extended reality integrated STEM curriculum could cooperate with various 
areas to design interdisciplinary curricula, add tasks and challenges with definite goals for students’ learning, as well 
as deepen the learned knowledge and apply the learned through cross-field collaboration and hands-on to achieve 
better learning effect. Lecturers would encounter resource problems when proceeding extended reality-integrated 
STEM curriculum, which requires more resources like field, equipment, and classroom facilities. A university should 
be able to provide resources to develop the curriculum. In this case, the universities should support STEM teach- 
ing for complete and adequate curriculum development. Moreover, team support and understanding of various 
teaching methods are required to develop such curricula to achieve the objective and effectiveness more easily. 

It is not easy to search for topics for an extended reality integrated STEM curriculum, the curriculum design 
would take much time, and the preparation is time-consuming, including the confirmation of course topic, activity 
process, and material purchase and preparation. It is therefore suggested that lecturers should well apply teamwork, 
group the curriculum team, and collaboratively prepare and precede the curriculum. Although it would take much 
time, the students’ reflection shows the curriculum’s benefit, revealing its positive effect. 


Limitations 


This study has limitations that should be noted. Firstly, the results cannot be generalized to a wider population 
due to the limited number of participants and the specific intervention used. This issue is common in educational 
research. Secondly, the study only looked at short-term retention of extended reality teaching. A longer-term 
evaluation would have provided more insights into the effectiveness of the STEM activities. Additionally, the lec- 
turer who taught the extended reality course was the same person who taught the control group. The lecturer's 
educational background and their use of extended reality in STEM teaching can impact the outcomes. Thirdly, the 
study had a limited number of participants, and a larger study with more participants is needed to draw more 
conclusive results. It is important to mention that all participants volunteered to take part. Finally, the study found 
that extended reality STEM activities were more effective than teaching in a control group, such as textbooks and 
digital films. However, these activities’ efficacy has some limitations, as discussed above. 
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Introduction 


Visual materials, such as illustrations and photographs, are not merely 
decorative or auxiliary elements in science textbooks. They play a central 
role in learning and are crucial for explaining scientific concepts (Ametller & 
Pint6, 2002; Pozzer & Roth, 2003). Scientific concepts are abstract and rooted 
in phenomena that students may find challenging to directly experience or 
observe. Consequently, scientific concepts cannot be effectively conveyed 
solely through linguistic information (Ametller & Pint, 2002). When students 
encounter such abstract concepts presented only in textual form, without 
accompanying visual materials or experiences, they may generate multiple al- 
ternative concepts (Roth et al., 1999). However, when illustrations are included 
alongside the textual content, they significantly contribute to understanding 
scientific concepts (Pozzer & Roth, 2003). In addition, the use of illustrations 
to explain scientific concepts proves valuable for learning comprehension 
due to their ability to represent multiple relationships and processes that are 
challenging to express through linguistic information alone (Meneses et al., 
2018; Vavra et al., 2011). 

Teaching astronomy, which belongs to the macroscopic domain of 
science, presents challenges due to the inability of students to directly ob- 
serve and fully perceive astronomical phenomena (Kim et al., 1998). To teach 
concepts about astronomy, textbook exploration activities often incorporate 
role-playing and model experiments. However, it is common for illustrations 
to be presented in the post-activity stage when summarizing the concepts. 
Simulations of celestial bodies’ motion in the context of learning astronomi- 
cal concepts often rely on elements presented in illustrations (Padalkar & 
Ramada, 2008). Thus, illustrations play a crucial role in the astronomy domain 
(Pena & Quliez, 2001). In astronomy illustrations, symbols such as arrows 
are frequently used to represent the direction of motion of celestial bodies. 
This inclusion of visual symbols is beneficial for students’ concept formation 
(Cho et al., 2009). 

In recent years, the importance of visual materials in educational re- 
sources has grown significantly, leading to a higher proportion of illustrations 
in textbooks and other teaching and learning materials (Lee & Kim, 2018; 
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Lowe, 2003; Schnotz, 2002). Nevertheless, despite the increased presence of illustrations, learners focus more on 
the text rather than the illustrations to aid their understanding of the material (Liu & Chuang, 2011; Pozzer & Roth, 
2003). Hannus and Hy6néa (1999) and Shin (2017) revealed that learners primarily allocate their attention to the 
text, spending less than 10% of their time looking at illustrations. In particular, primary school students perceive 
text as more important than illustrations when learning scientific concepts (Shin, 2017). 

However, to facilitate the understanding of scientific concepts, it is more effective to view text and illustrations 
together in an integrated manner (Mayer, 2003; Schnotz, 2002). Mason et al. (2013) reported that learning comprehen- 
sion is higher in groups where the ratio of integrated connections between text and illustrations is higher in illustrated 
science text learning. However, when primary school students learn from science textbooks, there is a low ratio of 
integrative connections between text and illustrations (Yang & Kim, 2013). 

To cope with this learning situation, teacher guidance and appropriate feedback are crucial to help students 
perceive the illustrations and text in an integrated manner (Scheiter & Eitel, 2015; Yang & Kim, 2013). In particular, in 
many cases, teachers may not always have control over the design of learning materials, and students may encoun- 
ter poorly designed materials in real-life learning situations. To assist learners in this situation, more interventions 
that employ generic methods are required rather than solely teaching them how to approach the material (Renkl & 
Scheiter, 2017; Scheiter et al., 2017). Recently, Eye Movement Modeling Examples (EMMEs) have emerged as one such 
intervention method (Mason et al., 2015, 2016; Mason et al., 2017; Scheiter et al., 2017). Studies have demonstrated 
that groups exposed to EMMEs exhibit a higher number of eye movement transitions between related texts and 
figures, resulting in enhanced learning comprehension. Another intervention method called Prompting involves 
providing another learning method in reading strategies, either verbally or in written form (van der Meij & de Jong, 
2011).When reading strategies are guided by Prompting, learners are more inclined to look at figures in text learning 
with illustrations, thereby improving their learning comprehension (Jian, 2019; Stalbovs et al., 2015). Accordingly, this 
study aimed to compare and analyze the effects of EMME, a reading strategy guide, and Prompting, a direct reading 
strategy guide, on primary school students (In Korea, this applies to students in grades 3-6.). The research questions 
of this study were as follows: 

1. What are the characteristics of primary school students’ reading process while learning science texts with 
illustrations? 

2. Istherea transfer effect when applying learner-centered intervention methods such as EMME and Prompt- 
ing during the learning process of science texts with illustrations? 

3. How does the application of EMME and Prompting affect students’ learning comprehension? 


Research Methodology 
Research Procedures 


The research began by formulating a specific research problem based on the literature review. Science texts 
were selected for research, and EMME videos and Prompting forms were developed to facilitate learning. The content 
validity of the selected science texts, EMME videos, and Prompting forms was assessed by three experienced science 
education experts who were well-versed in eye-tracking research. In addition, the tasks developed underwent refine- 
ment through preliminary research. Three types of tasks were devised for the study: a task to test the homogeneity 
of the group, an intervention-applied task, and a task verifying the effects of the intervention). Initially, a pre-test was 
performed to ensure that participating learners exhibited homogeneity within the group. Thereafter, the learners 
were exposed to the intervention through the task designed for intervention application. Finally, the science textbook 
focusing on ‘the rotation of the earth’ was selected as the final task and the effect of ‘intervention’ was evaluated. 


Sample Selection 


The recruitment of research participants took place in three primary schools located in a small town in the capital 
area of Korea. Fifth-grade students without difficulties in eye tracking were invited to participate in the study. Consent 
forms from students and their parents were collected, and those who provided the necessary consent became the 
research participants. The final sample for the study comprised 106 fifth-grade primary school students. Fifth-grade 
students were selected due to their developmental stage in reading, as indicated by Chall’s (1983) stages of reading 
development. During this stage, learners begin to form reading strategies and read for learning purposes. 
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The study comprised three groups: a group whose reading strategies were guided by EMME as an interven- 
tion method, a group whose reading strategies were directly guided by Prompting, and the control group (a group 
who was not intervened). Due to the use of an eye tracker in the EMME group, 19 participants were excluded. These 
exclusions included participants who failed to calibrate their gaze, those with a tracking ratio below 80% on each 
task, and those who had preconceptions of the learning task. Thus, the final analysis totaled 87 participants, with 29 
participants in each of the EMME, Prompting, and control groups. This study analyzed human eye movements and 
was pre-approved by the Institutional Review Board of the researcher's university. 


Development of Tasks 


Three tasks were developed and used in this study: a task to test the homogeneity of the group, an intervention- 
applied task, and a task to verify the effects of the intervention. The criteria for selecting the tasks were as follows: (1) 
choosing a topic that the learners had no preconception of, (2) ensuring that the illustrations were complementary 
to the text according to Pozzer and Roth's (2003) criteria, and (3) selecting tasks from the geoscience domain where 
illustrations are highly useful. These criteria were chosen because a preconception can affect eye movement (Can- 
ham & Hegarty, 2010). In addition, the study aimed to guide reading strategies that integrated illustrations and text. 
Therefore, the tasks were specifically chosen to ensure a complementary relation between illustrations and text. 

Considering the nature of primary school students’ thinking, illustrations are more useful in geoscience due to 
the high percentage of abstract representations (Pena & Quliez, 2001; Woo et al., 1992). Thus, all tasks were developed 
for the geoscience domain. The content validity of the developed tasks was confirmed by three experienced science 
education experts in eye-tracking research. The tasks that were finally developed were “Why wind blows” (task to test 
the homogeneity of the group), “Sea breeze and land breeze” (Intervention-applied task), and “Earth's rotation” (task 
verifying the effects of the intervention) (Figure 1). 


Figure 1 
Developed Tasks 


Air has weight. The pressing force generated by the weight of air is called atmospheric 
pressure. The greater the number of air particles in a certain volume, the heavier the air 
becomes and the higher the air pressure. Cold air has more air particles in a certain volume 
than warm air, so the air is heavier, and the air pressure is higher. This relatively heavy air is 
called high pressure, and light air is called low pressure. 

If there is a difference in air pressure between any two points, air moves from high pressure to 


low pressure. The movement of air due to the difference in atmospheric pressure is called 


wind. 
Hot air 
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a. Task to test the homogeneity of the group 
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On a clear day, the direction of the wind blown during the day and at night is different at the beach. 
What causes the wind to change direction during the day and at night at the beach? During the day, 
the temperature of the land and the sea increases due to sunlight. At this time, the land is hotter than 
the sea because it heats up faster. Because the temperature on land is higher than that of the sea, it 
creates low pressure on land and high pressure on the sea. During the day, the wind blows from the sea 
to the land as air travels from high pressure to low pressure. This wind from the sea to the land is called 
a sea breeze. 


At night, the temperature of the land and the sea decreases. Since land cools faster than the sea, the 
sea temperatures are higher than the land temperatures. Because the temperature of the sea is higher 
than that of the land, it creates low pressure above the sea and high pressure above the land. Therefore, 
the wind blows from the land to the sea. This wind from the land to the sea is called a land breeze. In 
this way, due to the temperature difference between the land and the sea during the day, the direction 
of the wind changes during the day and at night at the beach. 


A Daytime breeze (sea breeze) A Night breeze(land breeze) 


b. Intervention-applied task 


The Earth is moving. How does the Earth that we live on move? The Earth rotates once a day. It revolves around a 


virtual straight line connecting the North Pole and the South Pole, which is called the Earth's axis of rotation. 


The Earth's daily rotation from west to east is around its axis. The Earth rotates counterclockwise when viewed 


from above the Earth's north pole. 


Because the Earth rotates from west to east, it 
seems to us that the sun is moving from east to 
west. The side of the Earth that receives sunlight 
becomes day, and the side that does not receive 
sunlight becomes night. Day and night change 


periodically because of the Earth's rotation. 


A Earth's rotation and day and night 


c. Task verifying the intervention’s effects 


Content validity was verified for the following: @ Is the form of the task appropriate for learning? @ Is the 
amount of learning appropriate for a fifth grader? @ Is the task free of content errors? Specifically, the content 
validity index (CVI) for science education experts was 0.90 for the task on “Why wind blows (task to test the ho- 
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mogeneity of the group)” and 0.89 for the tasks on “Sea breeze and land breeze (intervention-applied task)” and 
“Earth's rotation (task to test the effects of intervention).’ 

In terms of task development, the final task on “Earth’s rotation” was designed to test the effects of the inter- 
vention. Unlike the other two tasks, the illustrations in this task were positioned at the bottom left rather than at 
the bottom of the text. This placement aimed to examine whether the transfer effect of reading strategies occurred 
regardless of the arrangement of illustrations and text. The study also examined the formation of preconceptions 
among typical fifth-grade students regarding the “Earth’s rotation” task. The findings indicated that only 1 in 25 
students had preconceptions about “Earth's rotation,’ suggesting that the influence of preconceptions on the study 
results was unlikely to be significant. 

The study aimed to compare the effects of two learner-centered interventions, namely EMME and Prompt- 
ing, on learners’ learning situations using eye tracking. A crucial aspect of the research was the development and 
application of EMME videos. In this study, EMME videos were created to deliver a reading strategy promoting the 
integrated reading of related illustrations and text. These videos were developed by modeling the teacher's (re- 
searcher’s) eye movements when viewing illustrated science texts. To develop the video, an eye tracker was used 
to record the teacher's eye movements while learning the treatment task “Sea breeze and land breeze,’ which was 
previously selected for intervention application (Figure 2). In this case, the researcher performed the task in a way 
that allowed the delivery of the reading strategy in a didactic manner after becoming familiar with the integrative 
reading strategy. 


Figure 2 
EMME video screen 


Indication of the gaze's location 


On a clear day, the direction of the wind blown during the day and at night is different at the beach. 
What causes the wind to change directign during the day and at night at the beach? During the day, 
the temperature of the land and the sea increases due to sunlight. At this time, the land is hotter than 
the sea because it heats up faster. Because the temperature on land is higher than that of the sea, it 
creates low pressure on land and high pressure on the sea. During the day, the wind blows from the sea 
to the land as air travels from high pressure to low pressure. This wind from the sea to the land is called 
a sea breeze. 


At night, the temperature of the land and the sea decreases. Since land cools faster than the sea, the 
sea temperatures are higher than the land temperatures. Because the temperature of the sea is higher 
than that of the land, it creates low pressure above the sea and high pressure above the land. Therefore, 
the wind blows from the land to the sea. This wind from the land to the sea is called a land breeze. In 
this way, due to the temperature difference between the land and the sea during the day, the direction 
of the wind changes during the day and at night at the beach. 


A Daytime breeze (sea breeze) A Night breeze(land breeze) 


For the development of the EMME video, this study analyzed previous research on integrated reading strategies 
for text and illustrations and adopted the following two perspectives: First, the researcher asked the participants 
to make connections between relevant pictorial factors when reading the text or to integrate the parts of the text 
related to the illustrations with the text. This was based on Mayer's Cognitive Theory of Multimedia Learning, which 
suggests that alternating and integrating the viewing of related text and figures enhances learning comprehen- 
sion in multimedia contexts (Mason et al., 2013; Mayer, 2003). Second, based on the work of Mason et al. (2017), 
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the researcher gazed at the illustrations and their descriptive captions before reading the text. In a previous study 
on integrated text and figure viewing using EMME (Mason et al., 2017), different groups were assigned to figure- 
first, text-first, figure-last, and control conditions to determine the effectiveness of delivering integrated reading 
strategies. The figure-first EMME group, which emphasized looking at the figure before reading the text, exhibited 
the highest rate of integrated eye movement. The post-test results indicated a statistically significant difference 
when compared with the control group. The EMME videos were developed based on the results of these previous 
studies. Additionally, before viewing the EMME task, students were informed about EMME through the instruc- 
tion that said, “The red dots in the video are the eye (gaze) movements of a good student. Study by fixating your 
gaze on the red dots in the video.’ The term “student” was used in the instruction to help students recognize the 
similarity between themselves and the model in the EMME video. Previous research has indicated that for novice 
learners, the effectiveness of observational learning with EMME increases when there is a higher similarity between 
the model and the observer (Krebs et al., 2019). 

Prompting tasks to guide reading strategies for integrating text and illustrations were developed by select- 
ing one for each of the reading strategies for text processing, figure processing, and integration presented in the 
study by Stalbovs et al. (2015) (Table 1). 


Table 1 
Reading Strategies used to Develop Prompting Tasks 


Multimedia course Directions 


Text processing When you're done, reread every paragraph carefully (McNamara et al., 2004; Zwaan & Singer, 2003). 


When looking at a figure, look for important parts of the figure that are relevant to your learning (Hegarty & 


Niustration processing Just, 1993: Hegarty & Sims, 1994). 


When reading the text, try to find the parts described in the text in the figure (Mason et al., 2013; Mayer, 


Integration 2009). 


Test Tools 


A questionnaire was developed and used to assess the learned content to determine if the learner-centered 
intervention was effective for learners’ science learning. The questionnaire comprising three items was developed 
for this study, fitting on a single sheet of A4 paper. The questionnaire was developed based on the achievement 
criteria outlined in the curriculum. The content validity of the developed questionnaire was verified twice by three 
science education experts who had experience in conducting eye tracking research and working in primary schools. 
The final CVI value was .87. The developed test was administered after the “Earth's rotation’ task to verify the effects 
of the intervention. The questionnaire comprised open-ended items, with one item requiring the participants to 
indicate the direction of Earth's rotation, an item featuring a figure that illustrated how day and night occur, and 
an item where the participants were asked to describe Earth’s rotation. A pilot study was conducted with 13 fifth- 
grade primary school students to confirm that no issues existed in the questionnaire design. 


Task Paradigm and Data Collection 


The overall procedure of the study was as follows: pre-test, pre-task (Why wind blows, the task of testing the 
homogeneity of the group), intervention-applied task (Sea breeze and land breeze), intervention effect verifica- 
tion task (Earth’s rotation), and post-test. The entire process of assigning tasks was conducted on a per-student 
basis because of the need for eye tracking. The eye tracker used was the X2-60 Eye Tracker from Tobii. Since eye 
tracking can be distracted by environments such as ambient noise or illuminance (Bojko, 2013), the measurements 
were conducted in a soundproofed consultation room to minimize these. The paradigmatic organization of the 
pre-task is indicated in Figure 3. 
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Figure 3 
Pre-task Paradigm 


When you're done 
learning, 
press the keyboard 


Let's learn the 


following textbook 
content 


10 sec 


3 sec 


5 sec Self-pace 


The pre-task was followed by the intervention-applied task and the intervention effect verifying task, using the 
paradigm in Figure 4. The entire process of eye tracking was with the learner’s learning, that is, self-reading of the 
learner, which is essentially a high-level cognitive process. Hence, it was carried out so that the learners could learn 
according to their reading speed without setting a time limit for the reading process (van der Schoot et al., 2012). 

For the intervention application task, as indicated in Figure 4, the EMME group was allotted 2 minutes 
and 48 seconds to watch a video illustrating a series of processes demonstrating the teacher's reading strate- 
gies for an illustrated text material on the topic of sea breeze and land breeze. The Prompting group studied 
an illustrated text about sea breeze and land breeze at their own pace after being instructed, “As you read, 
look for the parts described in the text in the figures.’ The control group, which did not receive any interven- 
tion, followed the same procedure as the pre-task paradigm presented earlier, starting with the instruction 
“Let’s learn the following textbook content.’ 


Figure 4 
Paradigm of the Intervention-applied Task 
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After a 10-minute break at the end of the intervention-applied task, the task for verifying the effect of the 
intervention was conducted, as indicated in Figure 5. The intervention-applied task was identically conducted to 
the procedure applied to the pre-task and control group. After all the tasks were completed, students in the study 
participated in a post-test, a written assessment to determine their learning comprehension of the tasks. 
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Paradigm of the Intervention Effect-verifying Task 
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Data Analysis 


The eye-tracking data of the two groups (EMME, Prompting) and the control group following the learner- 
centered intervention were analyzed using Tobii Pro Lab software from Tobii Company. Of the 106 students who 
participated in the study, 19 students with preconceptions of the learning tasks and 19 students with eye-tracking 
rates below 80% for each task were excluded. Accordingly, data from 87 students (EMME: 29 people, Prompting: 
29 people, Control: 29) were analyzed. In the process of extracting the eye movement data to a file using Tobii Pro 
Lab software, a minimum eye fixation time of 100 ms was set based on previous studies on the reading process 
(Andrews et al., 2004; Jian & Wu, 2015; Jian, 2016). To do this, a fixation filter was set up using Tobii Pro Lab’s I-VT 
filter algorithm. 

In consultation with three science education experts, areas of interest (AOls) were set on a sentence-by- 
sentence basis, as in the study by Hy6nd et al. (2003). The AOls were set only for the parts related to the illustrations. 
As a result, 9 parts were analyzed for the pre-task: 1 to 5 articles, 1 to 2 figures, and 1 to 2 captions. In cases where 
sentences were line-broken, the eye-movement data were extracted into an Excel file and the eye fixations were 
modified for Text 1-1 and Text 1-2 to Text 1 and for Text 2-1 and Text 2-2 to Text 2. For the intervention-applied task, 
to see the effects immediately after the Prompting was presented, the AOI was set as a total of 10 parts, including 
Text 1-6, figures 1-2, and captions 1-2 as shown in Figure 6. In cases where sentence-by-sentence line breaks oc- 
curred, a similar process was applied in the Excel file, merging the eye fixation data for Text 2-1 and Text 2-2 into 
Text 2 and Text 6-1 and Text 6-2 into Text 6. 


Figure 6 
Set up Learning Task AOIs 
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a. Task to test the homogeneity of the group 
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On a clear day, the direction of the wind blown during the day and at night is different at the beach. 
What causes the wind to change direction during the day and at night at the beach? During the day, 
the temperature of the land and the sea increases due to sunlight. At this time, the land is hotter than 
the sea because it bet "i faster. Because the temperature on land is higher than poet af ite sea, it 
creates low press, lext1 d and high pressure on the sea. During the day, the wi rom the sea 


to the land as air travels from high pressure to low pressure. This wind from the sea t¢_Text3_! is called 


a sea breeze. 


At night, the temperature of the land and the sea decreases. Since land cools faster than the sea, the 
sea temperatures are higher than the land temperatures. Because the temperature of the sea is higher 
than that of the land, it creates low pressure above the s& le*t4 4h pressure above the land. Therefore, 


the wind blo 


wo _TextS Je land to the sea. This wind from the land to the Cats led a land breeze. In 


this way, due to the temperature difference between the land and the sea during the day, the direction 
of the wind changes during the day and at night at the beach. 
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A Daytime breeze (sea breeze) & Night breeze(land breeze) 


b. ntervention-applied task 


Text 1 
The Earth is moving. How does the Earth that we live on move? The Earth Get a day. It revolves around a 


( Text2 ) 
virtual straight line connecting the Ge the South Pole, which is called the Earth's axis of rotation. 


The Earth's daily rotation ee ee is around its axis. The Earth rotates counterclockwise when viewed 


from above the Earth's north pole. 


A Earth's rotation and day and night 


: 

Because the Earth rotates trom west to east, it 

seems to us that the sun is moving from east to 
. C lets) 6 . . 

west. The side of the Ear at receives sunlight 

becomes day, and the side that does not receive 

sunlight becomes night. Day and night change 


periodically because of the Earth's rotation. 


c. Task verifying the intervention’s effects 


To identify the cognitive processes involved in learning illustrated science texts, this study utilized the Lag 
Sequential Analysis method proposed by Bakeman & Gottman (1997) to analyze the reading strategies indicated in 
group reading tasks. Sequential analysis methods can be used to infer reading strategies in the context of learning 
illustrated text by analyzing the number of moves between AOls (Bakeman & Gottman, 1997; Jian, 2016). Sequential 
analysis is a statistical approach used to examine the significance of the order of transitions between consecutive 
areas in a series of observations (Hsu et al., 2019). The study utilized the sequential analysis method outlined by 
Hsu et al. (2019) and utilized a sequential transfer diagram to interpret reading strategies. Here, the study focused 
on the number of eye movement transitions between two AOls, following the sequential analysis algorithm, and 
repeated eye movements to the same AOI were ignored in the analysis (Bakeman & Gottman, 1997; Hsu et al., 2019). 
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To determine the effect of the learner-centered intervention on science learning, the analysis of the post-test 
results assessing learning outcomes was scored separately for the pictorial and verbal factors. To analyze the results, 
the factors were extracted and analyzed, as shown in Table 3. A maximum of 4 points were separately assigned 
for the illustration factors and text factors, resulting in a total of one point each. For these items, assessment item 
factors were selected in consultation with three science education experts. The post-assessments of 87 participants 
were evaluated by three raters, yielding an inter-rater agreement of Cronbach's a = .940 for the pictorial factor and 
.956 for the verbal factor. 


Table 2 
Assessment of Item Factors for the Post-test Items 


Pictorial factor Verbal factor 
Representation of rotation direction arrows Counterclockwise, from West to East 
Representation of West and East One lap per day 
Representation of the phenomenon of day and night The sunlit side is day, 
occurring the side without sunlight is night 
Representation of the slope of the rotation axis Circling around the axis of rotation is rotation 


Research Results 


Intergroup Homogeneity as Revealed by Pre-Task and The Features of The Reading Process in 
Fifth-Grade Students’ Science Learning 


One of the objectives of this study was to investigate the reading process of EMME and Prompting interven- 
tion methods on the eye movements of students. To verify this, it is critical to confirm that no difference exists 
between groups. To do so, the study analyzed eye movement data from the pre-task activity titled “Why the wind 
blows” before administering the EMME and Prompting interventions, respectively, using Bakeman and Gottman’s 
(1997) sequential analysis method. This study utilized an expanded version of the Data Analysis Tool (DAT) vI.7, 
developed by Jeong (2003), to conduct sequential analysis. Based on the frequency of gaze transition between 
the indicated AOls, a transformation probability matrix was created to extract the z-score matrix (tables 3-5). The 
z-values presented in Tables 3-5 exceeded 1.96 (p < .05), implying the statistical significance of eye movement 
transitions between the two AOls (Jian, 2016). 


Table 3 
Z-score Matrix Table for the Pre-task Learning Process of the EMME Population 


Text 1 Text 2 Text 3 Text 4 Text 5 Figure 1 Figure 2. Caption 1 Caption 2 


Text 1 - 4.08 0.11 -0.54 -1.25 0.35 -0.21 -1.02 -1.02 

Text 2 4.93 - 6.72 -1.09 -1.39 -0.96 -2.70 -1.64 -1.64 

Text 3 0.40 4.72 - 5.74 -0.59 -1.66 -1.72 -1.86 -1.86 

Text 4 -1.34 -1.15 3.40 - 5.33 -1.04 0.28 -1.79 -1.79 

Text 5 -1.78 -2.05 -1.82 1.84 - 5.01 -0.16 -1.61 -0.87 
Figure 1 0.70 -0.51 -1.20 -2.19 2.61 - 2.90 6.45 -1.11 
Figure 2. -0.47 -0.47 -2.47 -~0.27 -1.92 2.64 - -1.37 8.75 
Caption 1 -1.13 -0.40 -0.69 -1.39 -1.25 4.12 0.53 - -1.02 
Caption 2 -1.31 -1.51 -1.04 -0.13 -1.45 -1.97 5.76 2.64 


Note. Shading indication z > 1.96 (p < .05) 
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Table 4 


Z-score Matrix Table for the Pre-task Learning Process for the Prompting Group 


Text 1 Text 2 Text 3 Text 4 Text 5 Figure 1 Figure2. Caption1 Caption 2 
Text 1 - 5.02 -0.72 -1.07 -0.58 0.35 -0.41 -0.78 -1.04 
Text 2 7.36 - 4.96 -0.99 -1.43 -2.58 -2.09 1.12 -1.50 
Text 3 -1.67 3.95 - 7.36 -0.19 -2.56 -2.62 -1.32 -1.77 
Text 4 -1.47 -0.64 3.77 - 3.64 0.18 -0.88 -0.22 -1.64 
Text 5 -0.84 -2.34 -0.71 0.68 - 5.16 0.59 -1.22 -0.87 
Figure 1 -0.69 -1.05 -2.27 -1.64 1.53 - 4.95 2.99 0.03 
Figure 2. -1.21 -0.88 -1.55 -1.99 -0.11 2.31 - 2.14 6.80 
Caption 1 -0.80 -0.89 -0.91 -1.07 -0.85 1.20 2.29 - 1.13 
Caption 2 0.46 -0.97 -0.99 -0.10 -0.92 -1.16 4.04 -0.50 - 


Note. Shading indication z > 1.96 (p < .05) 


Table 5 
Z-score Matrix Table for the Control Group's Pre-test Course of Study 


Text 1 Text 2 Text 3 Text 4 Text 5 Figure 1 Figure 2. Caption1 Caption 2 
Text 1 - 3.95 0.36 -0.38 -1.48 -0.50 0.08 -0.80 -0.93 
Text 2 7.05 - 8.12 -2.51 -2.48 -2.22 -2.51 -1.33 -1.55 
Text 3 -0.76 6.41 - 5.67 -1.32 -2.15 -2.27 -0.91 -1.28 
Text 4 -0.76 -2.66 3.65 - 4.51 0.64 -1.12 -1.39 -0.86 
Text 5 -0.18 -2.20 -2.21 2.58 - 5.25 -0.17 -1.41 -0.89 
Figure 1 -2.36 -0.92 -2.00 -0.87 4.20 - 4.07 4.53 -0.56 
Figure 2. -0.34 -0.63 -1.14 -0.19 -0.53 1.06 - -0.24 6.17 
Caption 1 -1.07 0.24 -1.64 -1.66 -0.53 4.07 -0.56 - 1.67 
Caption 2 1.15 -1.53 -1.76 -1.78 -1.48 0.86 6.41 1.93 - 


Note. Shading indication z > 1.96 (p < .05) 
To identify eye movement tendencies, only statistically significant cases are represented by arrows in Figure 7, 


which is represented as a gaze transition diagram. The values shown next to the arrows represent the transforma- 
tion probabilities documented in Tables 3-5. 
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Figure 7 
Diagram of Gaze Transitions between Areas of Interest in the Process of Pre-task Learning by Group 
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As a result of completing a gaze transition diagram (Figure 7) through sequential analysis to determine the 
reading strategies by group, similar eye movement tendencies were observed during the learning of illustrated 
science texts. This finding confirms the homogeneity among the three groups, indicating that they exhibit com- 
parable reading tendencies during science learning. 

Based on the results above, the characteristics of the reading process of fifth-grade students can be character- 
ized as follows: First, there appeared to be no connection between texts and figures, indicating no bidirectional 
reading between the illustration and the text. Second, learners exhibited a higher tendency to regress to previous 
sentences in the text. The frequent regression to the previous paragraph can be viewed as learners monitoring 
their own understanding. It can be viewed as a behavior that results from the absence of preconceptions. 


Comparison of the Learner’s Eye Movements after Applying a Learner-Centered Intervention 


After applying EMME and Prompting as learner-centered intervention methods to the “Sea breeze and land 
breeze” task, these methods were applied to the “Earth's Rotation” task for EMME, Prompting, and control groups 
to check the transferability of the interventions. To analyze the transition effect, a z-score matrix was obtained by 
generating a probability matrix for transformation between areas (Table 6). Based on these, the reading strategy 
for each group was schematized, as shown in Figure 8. 


Table 6 
Z-score Matrix Table in the Learning Process of the Intervention Effect-verifying Task (earth’s rotation) by Group 


EMME Group Text 1 Text 2 Text 3 Text 4 Text 5 Text 6 Earth Caption Sun 
Text 1 - 4.64 -1.94 -0.83 -1.07 -0.79 -0.79 -1.10 0.18 
Text 2 1.60 - 3.47 -0.87 -2.14 -1.57 3.85 -1.00 -0.60 
Text 3 -1.05 2.16 - 0.49 -1.00 -1.26 1.49 1.73 -1.56 
Text 4 -0.79 -2.36 -0.08 - 3.20 -0.95 2.40 -0.45 “1.17 
Text 5 -0.44 -0.77 -1.31 -0.56 - -0.53 3.39 -0.74 -0.66 
Text 6 1.14 -1.61 -0.41 -0.77 0.12 - 1.23 0.06 1.52 
Earth 0.41 3.22 2.39 1.98 1.32 2.31 - 1.11 2.03 

Caption -0.65 -0.62 -1.28 0.50 -0.02 2.00 1.35 - 0.18 
Sun -0.61 -0.99 -1.82 -0.77 0.12 0.74 2.93 0.06 - 
Prompting Group _ Text 1 Text 2 Text 3 Text 4 Text 5 Text 6 Earth Caption Sun 
Text 1 - 3.94 1.83 -1.54 -1.40 -1.18 -1.68 0.00 -1.18 
Text 2 4.34 - 2.32 0.51 -1.62 -1.37 0.19 0.00 -0.42 
Text 3 -1.51 3.27 - 3.77 -0.25 -1.51 -0.80 -0.01 -1.51 
Text 4 -0.98 -1.47 0.51 - 5.10 0.23 -0.47 0.00 -0.98 
Text 5 -0.26 -0.57 2.14 -0.34 - -0.26 -0.51 0.00 -0.26 
Text 6 -0.80 -1.76 -1.46 -1.05 -0.95 - 5.30 0.00 0.62 
Earth 0.22 -0.89 -0.57 -0.60 0.45 Sy) - -0.01 3.75 
Caption -0.52 1.15 0.37 -0.68 -0.62 1.57 1.50 - -0.52 
Sun -0.59 -0.22 -1.07 -0.77 -0.70 -0.59 3.38 0.00 
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Control group Text 1 Text 2 Text 3 Text 4 Text 5 Text 6 Earth Caption Sun 
Text 1 - 6.59 -1.64 -1.32 -1.57 -1.23 -2.22 -0.45 -0.64 

Text 2 3.31 - 5.32 -0.88 -1.32 -1.61 0.49 -0.59 -0.84 

Text 3 -1.32 1.69 - 2.60 -0.72 -1.23 1.08 -0.45 -0.64 

Text 4 0.09 -2.57 -0.32 - 3.71 -0.98 1.31 2.79 -0.51 

Text 5 -0.56 -0.26 -0.97 1.42 - 3.70 -0.94 -0.19 -0.27 

Text 6 -0.75 -1.83 -1.31 -0.75 1.74 - 2.82 -0.26 2.59 
Earth -0.46 0.26 -0.56 0.48 -0.04 3.71 - -0.48 1.15 
Caption 0.00 -0.01 0.00 0.00 0.00 0.00 0.00 - 0.00 

Sun -0.28 -0.67 -0.48 -0.28 -0.33 -0.26 2.17 -0.09 - 


Note. Shading indication z > 1.96 (p < .05) 


Figure 8 
Diagram of Gaze Transition between Areas of Interest in the Learning Process of the Intervention Effect-Verifying Task (earth's 
rotation) by Group 
5 12 
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Control group 


When seeing the gaze transition diagram of the EMME group (Figure 8), it can be observed that the AOI in the 
illustration, “Earth,’is centered on Text 2, Text 3, Text 4, Text 5, and Text 6, which are connected by arrows. Arrowed 
connections between AOls indicate statistically significant eye movements. In other words, the EMME group was 
able to make an integration between the visual information of the illustrations and the linguistic information of the 
text. When considering the probability of gaze transition between areas, represented by arrows, the probability of 
going from Text 2 to Text 3 (.53) exceeded the probability of going from Text 2 to Earth (.33). This observation shows 
different results from the learners reading the text sequentially and then looking at the figures before the EMME 
intervention. After the EMME intervention, learners exhibited a cognitive process of locating the parts mentioned 
in the text in the illustrations. This is evidence that the intervention utilizing EMME impacted learning-integrated 
reading strategies between illustrations and text. 

In the Prompting group, the portion of text associated with the pictorial factor “Earth” demonstrated an in- 
tegrated reading strategy only in the last part of the text, “Text 6.’This was similar to the eye movement tendency 
to look at the figures after finishing the text when studying the pre-task prior to the intervention. Furthermore, as 
shown in Figure 8, this eye movement tendency is similar to the reading strategies of the control group, who were 
not informed of the intervention. The control group also showed connections only in “Earth” and “Text 6” and no 
integrated connections between illustrations and text. This shows that the Prompting intervention had no effect 
on learners’ learning of integrated reading strategies. 

When intervening with Prompting, no integrated reading strategies between the illustrations and text were 
indicated either in “Sea breeze and land breeze” immediately after the intervention. The study found no change 
in reading strategies immediately following the Prompting intervention and no effect on reading strategies in 
subsequent learning tasks to determine the transferability of the intervention. 


Analysis of the difference in Learning Comprehension between EMME and Prompting Interventions 


To assess the difference in learning comprehension by learner-centered interventions between groups, the 
tasks used were analyzed by dividing them into pictorial and language elements. 

First, a one-way ANOVA was conducted to determine if there was a statistically significant difference in post- 
test scores for the pictorial factors across groups (Table 7). The EMME group had the highest post-test score for 
the pictorial factors at 2.38, and the Prompting group had the lowest score at 1.24. This difference was statistically 
significant. Post-hoc tests revealed significant differences between the EMME and Prompting and EMME and 
control groups, as shown in Table 8, and no statistically significant differences were observed between Prompting 
and control groups. 
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Table 7 


Comparison of Post-test Scores for Pictorial Factors by Group (one-way ANOVA) 
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Group n M SD F(p) 
EMME 29 2.38 1.12 
Prompting 29 1.24 83 9.20** 
Control 29 1.45 1.24 
*p<.01,** p<.001 
Table 8 
Scheffé Verification Results of Post-test Scores for Pictorial Factors 
Group EMME Prompting Control 
EMME 1.14* .93* 
Prompting 21 
Control 


*p<.01,** p< .001 


Second, a one-way ANOVA was conducted on the post-test scores for the language element by group. The 
post-test score for the language element was highest for the EMME group at 2.24 and lowest for the control group 
at 1.00. A one-way ANOVA revealed statistically significant differences between the groups (Table 9). Post-hoc 
tests showed statistically significant differences only between the EMME and control groups, with no statistically 
significant differences between the other groups, as shown in Table 10. 


Table 9 


Comparison of Post-test Scores for Language Elements by Group (one-way ANOVA) 


Group n M SD F(p) 
EMME 29 2.24 91 
Prompting 29 1.62 1.08 12.01** 
Control 29 1.00 89 
*p<.01,** p<.001 
Table 10 
Scheffé Verification Results of Post-test Scores for Language Elements 
Group Prompting Control 
EMME 62 1.24** 
Prompting 62 
Control 


*p<.01,**p<.001 


The results showed that the EMME group had higher learning comprehension than the other two groups. 
Based on the above results, it can be interpreted that the EMME group exhibited integrated reading strategies 
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between illustrations and text during the learning process of the “Earth’s Rotation,’ forming a coherent mental 
representation. This resulted in successful learning compared with the other two groups. 


Discussion 


A science textbook encompasses illustrations along with the text. Compared with other subjects, text alone 
is insufficient to convey learning concepts (Ametller & Pintd, 2002). According to Slykhuis et al. (2005), textbook 
content consisting of illustrations and text is more effective when the illustrations and text are complementary. As 
such, it is an effective teaching strategy to encourage learners to have an integrated understanding of illustrations 
and text when learning science. In this regard, this study analyzed the effects of two learner-centered intervention 
methods, EMME and Prompting, to intervene in the learner's integrated reading of illustrations and text. 

Prior to the learner-centered intervention, pre-task results revealed that the fifth-grade students, the study 
subjects, did not read illustrations and text in an integrated manner. These reading tendencies observed in young 
learners align with the findings of Moore and Scevak (1997) and Jian (2016). Moore and Scevak (1997) utilized think- 
aloud protocols and confirmed that young learners do not integrate text and illustrations in a cohesive manner. 
Through eye tracking to map reading strategies using a sequential analysis method, a study reported that while 
adults exhibited bidirectional connections between illustrations and text, sixth-grade students indicated no con- 
nections between text and illustrations. In addition, Yang and Kim (2013) demonstrated similar results, indicating 
that all students had poor integration of linguistic and visual information, regardless of the learning style. 

However, students who viewed the EMME videos, one of the learner-centered intervention methods used in 
this study, were more likely to use the strategy of integrating illustrations and text than other groups. Moreover, 
they also achieved significantly higher scores than the other groups on the learning comprehension test admin- 
istered as a post-test. This superior performance was observed not only on the pictorial factors but also on the 
verbal factors of the test. These results suggest that EMME videos are a good intervention method to teach reading 
strategies to learners. According to Mayer (2014), when studying multimedia materials consisting of text and figures, 
mental representation is necessary for learning. When learning from multimedia materials, comprehension of the 
content implies that the learner has a good mental representation of the text and figures through an integrated 
understanding. EMME helps learners’ mental representation through an integrated understanding of text and 
figures. The EMME intervention-induced changes in reading strategies identified in this study are consistent with 
the findings reported in prior research (Mason et al., 2015, 2016; Mason et al., 2017; Scheiter et al., 2017): EMME 
interventions increase eye movements in integrated reading when learning with multimedia materials consisting 
of text and illustrations. 

EMME videos show eye movements of a person (usually an expert) as they perform a task (Mason et al., 2015; 
Mason et al., 2017). EMME is an example-based learning, which is related to Sweller’s (1988) cognitive load theory. 

From a cognitive load theory perspective, providing step-by-step solutions with example-based learning 
helps learners create schemas for problem-solving, thereby helping them store schemas for problem-solving in 
long-term memory (Sweller et al., 2011). The resulting troubleshooting schema is used when encountering similar 
issues in the future. It supports the learner’s representational process for problem-solving. According to cognitive 
load theory, example-based learning reduces extrinsic cognitive load. It provides working memory capacity for the 
germane load associated with learning (Pass & van Gog, 2006; Renkl, 2011). Consequently, EMME can help learners 
learn by lowering their cognitive load and making efficient use of their working memory capacity. 

In addition to the EMME method, this study also used Prompting as a learner-centered intervention method. 
However, the Prompting method was not as effective as EMME and instead produced results similar to the control 
group that received no intervention. It was observed that the EMME method was more effective than the Prompt- 
ing method, which uses written or verbal prompts to guide learning because it uses the gaze of an expert or a 
higher-level learner to show learning strategies. 

Previous studies using EMME to study changes in learners’ eye movements have mostly focused on middle- 
school students (Mason et al., 2015, 2016; Mason et al., 2017) and college students (Scheiter et al., 2017). Accordingly, 
the results of this study suggest that applying EMME interventions can influence cognitive processes and reading 
strategies even in primary school students who have not yet formed reading strategies. 

In addition, the topic of “Earth’s rotation” was selected as a learning task in this study. The concept of “Earth’s 
rotation” has been associated with various misconceptions among young learners (Plummer et al., 2011; Plum- 
mer et al., 2014). Prior research on Earth’s rotation has suggested that consistent presentation of illustrations in 
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textbooks is effective for concept acquisition and retention (Cho et al., 2009). Cognitive processes for illustrations 
are critical because learners construct simulations based on pictorial factors presented in illustrations (Padalkar 
& Ramada, 2008). 

In this regard, the higher post-test scores of the EMME group on the pictorial factors and verbal factors com- 
pared with the other two groups implied that the integrated reading strategy between the illustrations and the 
text influenced learning comprehension of “Earth's Rotation.’In other words, when learning about “Earth’s rotation,’ 
a concept for which learners have many misconceptions compared with other concepts, the cognitive process of 
providing illustrations in a learning process such as EMME can be an important learning variable. 

Science education should develop inquiry ability and reasoning in students, rather than the simple acquisi- 
tion of scientific knowledge, and support their experience of creative problem-solving. EMME can help facilitate 
the learning of beginners. EMME is useful for solving various tasks such as observation and classification (Jarodzka 
et al., 2010), problem-solving (van Gog et al., 2009), and synthesis and analysis of materials (Litchfield et al., 2010). 
The use of EMME facilitates the visualization of the perceptual process and cognition of experts and the education 
of reasoning based on visual observation. 

Furthermore, the materials used by students in school comprise highly visualized materials, unlike the past 
when they were text-centered. As students advance to higher grades, the materials become more diverse, so the 
ability to understand texts and illustrations in an integrated manner will be even more emphasized. While students 
acquire information in order to use materials like textbooks, they will approach several complex materials, such 
as hypertext, in a nonlinear manner in the future. Thus, acquiring information processing methods using visual 
observation technology through EMME will be useful in science education hereafter. 


Conclusions and Implications 


Students primarily acquire knowledge in school through reading. Illustrations in science textbooks play a 
crucial role in conveying important information. To promote students’ integrated understanding of illustrations and 
texts, a learner-centered intervention method is necessary. The purpose of this study was to analyze the learning 
process of learners following a learner-centered intervention method using eye trackers and examine their learning 
comprehension. Based on the study findings, the following conclusions can be drawn: First, when primary school 
students were exposed to observational learning using EMME, which incorporated the teacher's gaze movement 
information, it had an impact on their cognitive processes and reading strategies during science learning. The 
reading strategies exhibited in the process of learning an illustrated science text (specifically, Earth's rotation) were 
analyzed through eye movements and diagrams. Compared with the other groups, only the EMME group dem- 
onstrated a connection between the pictorial factor “Earth” and the sentences associated with the illustration. A 
change in reading strategies was observed after the EMME intervention. Second, the EMME intervention improved 
learning comprehension in science learning. The EMME group, which demonstrated integrated reading strategies 
between illustrations and text, scored higher on the post-learning test compared with the other two groups, and 
this difference was statistically significant. These findings confirm the influence of the integrated reading strategies 
of illustrations and text on learning comprehension. 

Based on this research, the following suggestions can be made for science learning in schools: When guid- 
ing primary school students through science learning with a learning intervention, they should be instructed 
through integrated reading strategies that incorporate illustrations and text presented in the learning materials. 
Moreover, guided reading strategies, such as the EMME learning interventions identified in this study, are more 
effective when they showcase adults’ reading process through observational learning rather than solely presenting 
linguistic information. Since the teacher's presentation of linguistic information alone is insufficient to influence 
learners’ reading strategies, methods such as EMME, which demonstrate a series of concrete processes, should be 
considered to provide appropriate feedback. 
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Abstract. Several studies compared three 
different types of questions (conceptual, 
algorithmic, and graphical) across various 
topics, however, few focused specifically 
on gifted students. This study addressed 
this gap. The aim of the study, hence, 

was to determine whether there were 
notable differences in gifted students’ 
performance in the three types of tests. 
The study involved 115 gifted students 
aged between 17 and 18 years old. They 
responded to the three different tests 
including 10 test items in each with the 
same content. Significant differences were 
observed in their performances between 
the conceptual versus algorithmic in favour 
of the conceptual test, the conceptual 
versus graphical in favour of the conceptual 
test, and the algorithmic versus graphical 
in favour of the algorithmic test. Moreover, 
the statistical analysis results revealed 
that the students performed markedly 
poorer performance on the graphical test 
in comparison to both the algorithmic and 
conceptual tests. These results suggest 

the need to enhance students’ graphical 
skills to facilitate a better understanding 
of physics concepts. Proper steps should 
be taken to improve their proficiency in 
interpreting and analysing graphical 
representations. 
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GIFTED STUDENTS’ 
PERFORMANCE ON 
ALGORITHMIC, CONCEPTUAL, 
AND GRAPHICAL QUESTIONS 


Fatma Costu 


Introduction 


Scientific knowledge can be communicated through various forms of 
representation. They are mostly given as written texts, diagrams, simulations, 
graphs and so forth in the relevant literature (Airey & Linder, 2009; Guo, 
2004). Each of these forms of representation possess unique characteristics 
that allow them to convey specific information and meaning (Lemke, 1990). 
Researchers (Jaipal, 2010; Kress et al., 2014; Tang et al., 2011, Tsui & Treagust, 
2013) have explored the connection between the construction of scientific 
knowledge and the development of different types of representation. They 
have also investigated the cognitive requirements for learners to understand 
and make sense of these representations. In particular, the semiotic properties 
of graphs have been extensively studied by several researchers. It has been 
noted that graphs encompass both linguistic and symbolic elements and 
can be utilized for logical reasoning between variables or elements within 
a given knowledge unit (Tsui & Treagust, 2013). Consequently, interpreting 
graphs necessitates cognitive processes that differ from those involved in 
understanding linguistic elements (Tang et al., 2014). In physics learning, 
the comprehension of multiple forms of representation is often crucial, and 
conceptual learning is facilitated by utilizing the representations that provide 
the highest affordance. The ability to interpret graphs is a complex skill that 
requires contextual knowledge and the ability to translate unfamiliar graphs 
into verbal descriptions. This task can even challenge scientists who are 
unfamiliar with the specific subject matter (e.g., Guo, 2004; Tsui & Treagust, 
2013). Thus, graph interpretation goes beyond mere visual understanding 
and entails a deeper understanding of the underlying concepts and their 
relationships. 

Based on various forms of representation, different types of questions 
have been identified in the literature. These include conceptual questions 
(e.g., Nurrenbern & Pickering, 1987), algorithmic questions (e.g., Nakhleh & 
Mitchell, 1993), graphical questions (e.g., Costu, 2007; 2010; Kurnaz, 2013), 
context-based questions (e.g., Kaltakci & Eryilmaz, 2011), and pictorial ques- 
tions (e.g., Habiddin & Page, 2021). Numerous studies have been conducted 
to determine which types of questions students excel in. In the chemistry 
education literature, the initial comparison between conceptual and algorith- 
mic questions was conducted on gas laws and stoichiometry, which revealed 
that most university students answered algorithmic questions correctly but 
struggled with conceptual questions (Nurrenbern & Pickering, 1987). This find- 
ing has been confirmed by many subsequent studies in chemistry education 
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(e.g., Cracolice et al., 2008; Demircioglu & Ercebi, 2013; Gultepe et al., 2013; Lin et al., 1996; Nakhleh, 1993; Nakhleh 
& Mitchell, 1993; Papaphotis & Tsaparlis, 2008; Park et al., 2011; Pickering, 1990; Stamovlasis et al., 2005; Yilmaz et 
al., 2005), indicating that most students show a preference for algorithmic questions over conceptual ones. Similar 
trends have been observed in physics education, although to a lesser extent (e.g., Costu & Satilmis, 2020; Duizgtin 
et al., 2001). In both physics and chemistry, there is a general tendency for students to focus on solving algorithmic 
questions without a sound understanding of the underlying concepts, although there are some contradictions to 
this trend. Over the years, several studies have compared students’ performance on conceptual, algorithmic, and 
graphical questions in the fields of chemistry (e.g., Costu, 2007, 2010), physics (e.g., Costu & Satilmis, 2020; Kurnaz, 
2013), and mathematics (e.g., Incikabi et al., 2015). These studies have shown that developing a “graphical under- 
standing’ is crucial for students to better comprehend these subjects. When comparing students’ achievements on 
conceptual, algorithmic, and graphical tests, the results consistently indicate that students tend to perform better 
on algorithmic and conceptual tests but struggle with graphical tests. Furthermore, the emphasis on mathematics 
in chemistry and physics education can be attributed to the fact that mathematical representations have been an 
integral part of these subjects for a longer period (Schuchardt & Schunn, 2016; Steen, 2005). 

As evident from the aforementioned literature, studies comparing students’ performance in conceptual, 
algorithmic, and graphical questions have been conducted at the middle school, high school, and university 
levels. However, there is little research on gifted students in this area. Nonetheless, there is a growing recognition 
that gifted students need to develop skills in mathematical symbol reasoning and graphical thinking, given the 
rapid advancements in research technologies. Several papers have highlighted the importance of nurturing these 
skills among gifted students to enhance their understanding of complex scientific concepts and foster their suc- 
cess in science. For instance, Kurnaz (2018) conducted a study that demonstrated a strong correlation between 
mathematical and spatial skills and students’ success in the field of science. These findings suggest that fostering 
mathematical and graphical reasoning skills should be a crucial component of gifted students’ education to en- 
hance their abilities. One approach to facilitate the development of gifted students’ abilities is to tailor education 
to their specific needs, thereby motivating them to further enhance their talents (e.g., Dai & Chen, 2013; Kapici & 
Costu, 2023). Providing appropriate challenges can increase their motivation to learn (Phillips & Lindsay, 2006), 
as gifted students often enjoy tackling complex problems due to their advanced problem-solving skills (Kapici & 
Costu, 2023; Scager et al., 2013; Steiner & Carr, 2003). Based on these suggestions, there is aneed to determine the 
extent to which gifted students possess algorithmic, conceptual, and graphical skills, which are crucial for teaching 
science concepts. Additionally, examining the performance of gifted students across the three question types can 
provide further insights into their specific learning challenges. Therefore, this study focuses on gifted students and 
aims to assess their proficiency in algorithmic, conceptual, and graphical understanding. 

Therefore, the aim of the study was to examine whether significant differences exist in the performance of 
gifted students across three types of questions: namely “conceptual”, “algorithmic” and “graphical” The study ex- 
amined three research questions, which are as follows: 

1. Are there any significant differences in gifted students’ performance amongst conceptual, algorithmic, 
and graphical tests? 

2. Do the gifted students show the best, the moderate and the worst performance on which of the ques- 
tion types? 

3. Is there any positive connection amongst conceptual understanding, algorithmic understanding, and 
graphical understanding of the gifted students? 


Research Methodology 
Research Design 


A quantitative descriptive research design was employed in this study, with a focus on statistical analysis 
of data collected through three types of tests. The quantitative descriptive research design is an approach used 
in educational research to systematically collect and analyse numerical data in order to describe or summarize a 
specific phenomenon or group. This design is widely utilized to address research inquiries related to the preva- 
lence, patterns, and connection among variables within the field of education (Bloomfield, & Fisher, 2019; Cohen, 
Manion, & Morrison, 2017). 
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The study included a total of 115 gifted students, aged between 17 and 18 years old, who were enrolled in 
the Science and Art Center (SAC) at five different high schools (anonymous). The sample size for this study was de- 
termined based on the population of gifted students enrolled in the SAC at five different high schools. This sample 
size was deemed appropriate and representative of the target population, allowing for sufficient statistical power 
to detect meaningful differences and draw reliable conclusions. The selection of participants from multiple high 
schools aimed to enhance the generalizability of the findings. However, it is important to note that the specific 
process and criteria for participant selection were not included in this paper. Ethical considerations were followed, 
and participant anonymity was maintained throughout the study. In the country, SACs serve as educational in- 
stitutions specifically designed to offer advanced education in fields like science, mathematics, and art to gifted 
students (Sahin, 2013). These centres aim to foster the development of students’ abilities in their areas of interest 
and support their personal and social growth. The curriculum at SACs is specifically tailored to meet the unique 
needs of gifted students, offering engaging and challenging courses (e.g., Kahveci & Akgtl, 2014). Admission to 
SACs is based on a competitive entrance exam that assesses students’ academic abilities and potential. Those who 
perform successfully on the entrance exam are granted admission to SACs, where they receive education from 
qualified teachers and engage in extracurricular activities aimed at enhancing their skills and knowledge. As a 
result, the participants in this study share similar backgrounds and possess comparable knowledge in the topics 
relevant to the study. It should be noted that the participants had previously studied the topics addressed in this 
research, indicating their familiarity with the subject matter. 


Instruments 


To address the research questions, the study employed three tests: the Conceptual test (C), the Algorithmic 
test (A), and the Graphical test (G) (Costu, 2007; 2009; Kurnaz, 2013). Each test comprised ten multiple-choice items. 
These test items were developed by drawing from a variety of physics textbooks, theses, and question banks, 
specifically targeting the research topic outlined in Table 1. 


Table 1 
Specification of the Three Types of Test Items 


Test items Question peers* The content of the items 

Question 1 (1A-1C-1G) Relative Motion 

Question 2 (2A-2C-2G) Newton's Laws of Motion 

Question 3 (3A-3C-3G) One-Dimensional Motion with Constant Acceleration 
Question 4 (4A-4C-4G) Two-Dimensional Motion with Constant Acceleration 
Question 5 (5A-5C-5G) Energy and Motion 

Question 6 (6A-6C-6G) Repulsion and Linear Momentum 

Question 7 (7A-7C-7G) Torque 

Question 8 (8A-8C-8G) Simple Machine 

Question 9 (9A-9C-9G) Uniform Circular Motion 

Question 10 (10A-10C-10G) Simple Harmonic Motion 


Note. *“A, C and G” refers to algorithmic, conceptual and graphical, respectively. 


An example of each of the three question types (specifically, questions 4C, 4A, and 4G) is provided in the Ap- 
pendix. The study utilized three distinct types of question items. The first type, referred to as conceptual questions 
(4C, see Appendix), required students to demonstrate their understanding of the underlying concepts. Conceptual 
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questions assess students’ comprehension of the fundamental ideas and principles in a subject, rather than their 
ability to perform specific procedures or calculations. They aim to encourage critical and deep thinking, prompting 
students to apply their knowledge to solve problems and address questions, instead of relying solely on memo- 
rization of facts or procedures. 

The second question (4A, see Appendix), known as an algorithmic question, prompts students to utilize a 
predetermined set of rules or procedures to arrive at a numerical solution for the given problem. Algorithmic ques- 
tions involve applying established algorithms or procedures to solve problems or answer questions. Typically, these 
questions involve calculations or the manipulation of formulas and equations, assessing the student's proficiency 
in executing specific algorithms or procedures accurately. This question type aligns with the established definition 
of algorithmic questions found in the literature. 

The final question type (4G, see Appendix) involved the utilization of graphical knowledge, interpretation, 
and thinking. A graphical question refers to a question that necessitates students to interpret or analyse informa- 
tion presented in the form of a graph or diagram. It assesses the student's capacity to comprehend and employ 
visual representations to solve problems or respond to inquiries. This question category aligns with the established 
definition of a graphical question as outlined in the literature. 


Reliability and Validity 


The three tests underwent a pilot phase with a separate group of 30 gifted students, distinct from the par- 
ticipants in the main study. The internal consistency of each test was evaluated by measuring the reliability using 
Cronbach's Alpha coefficient. The results indicated a reliability coefficient of .75 for the conceptual test, .77 for 
the algorithmic test, and .71 for the graphical test. The difficulty index (p) of the test items ranged from .34 to .70, 
while the discrimination index (d) of each item was equal to or greater than .27. These results suggest that the 
tests exhibit satisfactory internal consistency and can be considered reliable measurement tools for comparing 
the performance of gifted students across the different test types. 

To ensure the validity of the tests, a panel consisting of two physics teachers and two science educators as- 
sessed the difficulty level of each test item and confirmed the overall validity of the tests. 


Data Analysis Procedure 


A total of 115 gifted students participated in the examination, completing all 30 test items within a single 
45-minute session. A scoring system was developed to assess the students’ performance, with incorrect or missing 
responses receiving a score of 0 points and accurate responses assigned a score of 1 point. The total scores for each 
student were calculated across the three tests. 

Statistical analysis was conducted to compare the students’ scores across the three test types. Descriptive 
statistics were initially performed to examine the distribution of scores for each test. The normality of the data was 
assessed using the Shapiro-Wilk test to determine whether parametric or non-parametric tests should be used. 
Based on the results of the normality assessment, either one-way ANOVA (for parametric data) or the Kruskal- 
Wallis H test (for non-parametric data) was employed to determine the significance of differences among the test 
types. Subsequently, multiple comparisons of the gifted students’ test scores were conducted to identify specific 
significant differences between each test and the other two. 

Furthermore, a specific set of criteria was utilized to compare the students’ performance on one type of item 
with their performance on the other two test types, allowing for a more detailed analysis of their performance 
across different question categories. 


Research Results 


The responses of the gifted students in the three question types are presented in Table 2. 
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Table 2 


Gifted Students’ Responses in Three Types of Test Items 


Question items 


1A-1C-1G 
2A-2C-2G 
3A-3C-3G 
4A-4C-4G 
5A-5C-5G 
6A-6C-6G 
TA-1C-7G 
8A-8C-8G 
9A-9C-9G 
10A-10C-10G 


Total 


Ai 
97 
77 
65 
100 
40 
91 
65 
94 
87 
108 
824 


Algorithmic test 
AO A- 
14 4 
27 11 
45 5 
8 7 
61 14 
22 2 
49 1 
19 2 
25 3 
5 2 
275 51 


C1 
103 
102 
42 
99 
82 
90 
92 
100 
96 
104 
910 
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Conceptual test 


co 


219 


20 


Graphical test 


Gi 
103 
65 
69 
93 
72 
28 
81 
104 
67 
50 
732 


GO 
10 
48 
45 
21 
37 
84 
33 
6 
47 
63 
394 


Note: A1, C1 and G1 refer to correct responses for algorithmic, conceptual, and graphical test items respectively. 


AO, CO and GO refer to incorrect responses for algorithmic, conceptual, and graphical test items respectively. 


A-, C- and G- refer to no response for algorithmic, conceptual, and graphical test items respectively. 


24 


/Print/ 


Based on the results presented in Table 2, it is evident that gifted students faced greater challenges with 
graphical questions, as a lower percentage of students provided correct answers compared to other question 
types. Similarly, they encountered difficulties with algorithmic questions, as a higher number of students did not 
respond to them compared to other question types. In general, the average of the correct scores for all test items 
was 71.6, with a standard deviation of 12.6. On average, the percentage of correct answers for each test type was 
as follows: algorithmic test items - 71.5 (SD = 18.8), conceptual test items - 79.0 (SD = 14.3), and graphical test 
items - 64.3 (SD = 15.6). 

To determine whether to use parametric or non-parametric tests, the normal distribution of the data was as- 
sessed using the Shapiro-Wilk test. The results indicated that all three tests followed a normal distribution (p = .22 > 
.05 for the algorithmic test, p =.11 > .05 for the conceptual test, and p =.18 > .05 for the graphical test). Therefore, a 
one-way ANOVA test was employed to compare the performance of gifted students on each test. Table 3 presents 


a statistical analysis of the differences in the results. 


Table 3 


Results of One-way ANOVA 


Between groups 


In groups 


Total 


Sum of squares 
12419.2 
92138.1 
104557.2 


df 

2 
343.5 
344.5 


MS 


6209.6 
134.1 


According to Table 3, there were significant differences in test scores (p = .0001 < .05). Based on the average 
achievement scores of all three test types, students performed the best in the conceptual test and the worst in the 
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graphical test. In other words, students demonstrated the highest level of performance in conceptual questions 
compared to all other question types. Additionally, a post hoc (Tukey HSD) statistical analysis was conducted to 
identify the tests that showed significant differences. The results of the post hoc analysis are presented in Table 4. 


Table 4. 
Results of Multiple Comparisons of the Students’ Test Scores (based on Tukey HSD) 


(D) Test (E) Test Mean Difference (D-E) In favour of (p) Sig. 
Conceptual (C) (A-C) -7.48* Conceptual .0001 
Algorithmic (A) 
Graphical (G) (A-G) 7.22* Algorithmic .0001 
Algorithmic (A) (C-A) 7.48* Conceptual .0001 
Conceptual (C) 
Graphical (G) (C-G) 14.70* Conceptual .0001 
Conceptual (C) (G-C) -14.70* Conceptual .0001 
Graphical (G) 
Algorithmic (A) (G-A) -7.22* Algorithmic .0001 


Note. * The mean difference is significant at the .05 level 


The results of the multiple comparisons revealed a significant distinction between the conceptual test and the 
graphical test (p = .0001 < .05), favouring the conceptual test. Similarly, significant differences were found between 
the conceptual test and the algorithmic test (p = .0001 < .05), favouring the conceptual test. Likewise, significant 
differences were observed between the graphical test and the algorithmic test (p = .0001 < .05), favouring the 
algorithmic test. To categorize students as high or low performers, a score of over 50% was considered indicative of 
high performance (coded as H), while a score of less than 50% indicated low performance (coded as L). The codes 
assigned to each test item were determined based on the student's performance on each question within the pairs. 
Figure 1 illustrates all possible comparisons between the three test types. A specific set of criteria was utilized to 
compare the students’ performance on the algorithmic test items with their performance on the conceptual and 
graphical test items. Figure 1 presents all potential outcomes of these comparisons. The distribution of the gifted 
students’ total score is depicted in Figure 1 as follows: 59% in the HAHC category, 20% in the LAHC category, 13% 
in the HALC category, and 8% in the LALC category. These findings suggest that the majority of students displayed 
the capacity to successfully apply relevant concepts when solving algorithmic problems. 


Figure 1 
Categories Based on Algorithmic Questions versus Conceptual Questions, Graphical Questions versus Conceptual Questions 


and Algorithmic Questions versus Graphical Questions 


Conceptual Question 


High (H) Low (L) 
8 Good at algorithmic test, Good at algorithmic test, 
z High (H) good at conceptual test poor at conceptual test 
2 (HAHC) 59% (HALC) 13% 
E Poor at algorithmic test, Poor at algorithmic test, 
& Low (L) good at conceptual test poor at conceptual test 
< (LAHC) 20% (LALC) 8% 
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SS Conceptual Question 
High (A) Low (L) 
S Good at graphical test, good | Good at graphical test, poor 
3 High (H) at conceptual test at conceptual test 
= (HGHC) 53% (HGLC) 11% 
r Poor at graphical test, good | Poor at graphical test, poor 
& Low (L) at conceptual test at conceptual test 
(LGHC) 26% (LGLC) 10% 
Graphical Question 
High (A) Low (L) 
& Good at algorithmic test, Good at algorithmic test, 
g High (H) good at graphical test poor at graphical test 
2 (HAHG) 48% (HALG) 23% 
: Poor at algorithmic test, Poor at algorithmic test, 
Sp Low (L) good at graphical test poor at graphical test 
(LAHG) 17% (LALG) 12% 


For a detailed breakdown of the students’ performance on each specific test item category (algorithmic test 
items versus conceptual test items), please refer to Table 5. The results presented in Table 5 reveal that gifted stu- 
dents in the HAHC category achieved the highest average percentage (68%) compared to the other performance 
categories. Conversely, students in the HALC category had a moderately lower average percentage (15%) among 
the other performance categories. This suggests that although these students performed well on algorithmic test 
items, they struggled with answering conceptual test items correctly. In summary, the categories of student per- 
formance are interconnected, and the results indicate that HAHC exhibited the highest percentages, while HALC 
showed slightly lower percentages, aligning with the statistical differences observed in Table 4. These consistent 
findings point to a positive correlation between the algorithmic abilities of gifted students and their conceptual 
understanding of physics topics. 


Table 5 
Percentages of Algorithmic Test Items versus Conceptual Test Items 


HC LC 

HA LA HA LA 
Question 1 86 16 10 3 
Question 2 69 33 7 6 
Question 3 23 19 42 31 
Question 4 87 12 12 4 
Question 5 31 52 9 23 
Question 6 70 20 21 4 
Question 7 54 39 1 11 
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HC Lc 

HA LA HA LA 

Question 8 83 17 1 4 

Question 9 73 22 15 5 

Question 10 99 5 9 2 

Average 68 23 15 9 


Regarding the performance of gifted students on the conceptual test versus the graphical test, Figure 1 il- 
lustrates the distribution of students’ total performance as follows: HCHG, 53%; HCLG, 26%; LCHG, 11%; and LCLG, 
10%. These results indicate that the most students had a stronger conceptual understanding and were able to 
interpret graphs related to physics concepts correctly (i.e, HCHG 53%). However, students who achieved LCHG 
performance had the slightly lowest average percentage (11%) among the other performance categories, sug- 
gesting that although these students struggled with conceptual test items, they also answered graphical test 
items incorrectly. This finding supports the notion that there is a positive correlation between the gifted students’ 
conceptual understanding and their graphical abilities. In other words, the stronger their conceptual understanding 
of physics topics, the greater their graphical abilities. For a more detailed analysis of each test item, the students’ 
performance on conceptual test items versus graphical test items is provided in Table 6. 


Table 6 
Percentages of Conceptual Test Items versus Graphical Test Items 


HC Lc 

HG LG HG LG 

Question 1 96 6 10 3 
Question 2 59 43 6 g 
Question 3 25 17 45 28 
Question 4 83 16 10 6 
Question 5 58 25 14 18 
Question 6 20 70 8 17 
Question 7 68 25 14 8 
Question 8 94 5 1 5 
Question 9 58 37 10 10 
Question 10 47 58 4 6 
Average 61 30 13 11 


As observed in Table 6, students performing at the HCHG level achieved the highest percentage in most ques- 
tions, with the exception of questions 3, 6, and 10. These findings indicate that students with strong conceptual 
understanding also tend to have a good grasp of graphical concepts, aligning with the results from the total scores 
presented in Figure 1. 

Regarding the students’ overall performance on the algorithmic test compared to the graphical test, there 
was a distribution of total performance among the students: HAHG, 48%; LAHG, 17%; HALG, 23%; and LALG, 12%, 
as shown in Figure 1. This distribution suggests that a majority of the gifted students were able to apply algorith- 
mic thinking to comprehend and interpret graphical questions related to physics. To examine this in more detail, 
a breakdown of the students’ performance on each individual item (pertaining to conceptual test items versus 
graphical test items) is presented in Table 7. 
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Percentages of Algorithmic Test Items versus Graphical Test Items 


HA LA 

HG LG HG LG 

Question 1 90 6 16 3 
Question 2 49 28 17 21 
Question 3 42 23 28 22 
Question 4 82 17 11 5 
Question 5 30 10 42 33 
Question 6 22 69 6 18 
Question 7 46 19 36 14 
Question 8 88 6 17 4 
Question 9 54 34 14 13 
Question 10 48 60 3 4 
Average 55 27 19 14 


As observed in Table 7, students who demonstrated HAHG performance achieved the highest percentage in 
most questions, with the exception of questions 5, 6, and 10, as reflected in the overall results presented in Figure 
1. This finding suggests a correlation between students’ algorithmic problem-solving skills and their understanding 
of graphs. In other words, most students were able to solve the problems and interpret graphs correctly. Further- 
more, the high percentages in the LG category for almost every question indicate that the graphical skills of the 
gifted students were significantly inadequate. 


Discussion 


The overall analysis of the gifted students’ performance on the three physics tests indicated a significant 
difference in their scores. Specifically, the students performed significantly worse on the graphical test compared 
to the conceptual and algorithmic tests. These findings align with previous studies conducted on chemistry 
topics (e.g., Costu, 2007; 2010) and physics topics (e.g., Costu & Satilmis, 2020; Kurnaz, 2013) involving non- 
gifted students. However, they are inconsistent with prior research on chemistry topics (e.g., Mason et al., 1997; 
Nakhleh, 1993; Niaz, 1995; Pickering, 1990; Pushkin, 1998; Sawrey, 1990) and physics topics (e.g., Duzgun et al., 
2001), which suggest that non-gifted students may employ algorithms without a significant level of conceptual 
understanding. These discrepancies in the literature may be attributed to students employing non-conceptual 
strategies when solving algorithmic questions in multiple-choice test items (Sung et al., 2022). 

For a comprehensive analysis, let’s address the first research question of the study. The results revealed 
statistically significant differences among the three test scores. The gifted students exhibited the highest perfor- 
mance on the conceptual test, while their performance on the graphical test was the lowest. Multiple comparisons 
indicated statistically significant differences between the conceptual and algorithmic test scores, favouring the 
conceptual test. Similarly, there were statistically significant differences between the graphical and algorithmic 
test scores, favouring the algorithmic test. Additionally, there were statistically significant differences between 
the graphical and conceptual test scores, favouring the conceptual test. These findings contradict previous 
studies conducted on chemistry (e.g., Mason et al., 1997; Nakhleh, 1993; Niaz, 1995; Pickering, 1990; Pushkin, 
1998; Sawrey, 1990) and physics topics (e.g., Duzgtin et al., 2001), but are consistent with more recent literature 
(Costu, 2007; 2010; Costu & Satilmis, 2020; Kurnaz, 2013) that suggests many non-gifted students are capable 
of solving algorithmic problems with conceptual understanding. One possible reason for these results could be 
attributed to giftedness (e.g., Godor, 2019; Kapici & Costu, 2023), as there exists a strong association between 
conceptual understanding and giftedness in science education. As discussed in the literature, gifted students 
are individuals with exceptional skills and intellectual abilities, often demonstrating above-average proficiency 
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in academic subjects such as math, science, and language (e.g., McCoach et al., 2001; Steiner, & Carr, 2003, 
Tosun, 2022). Furthermore, the study suggests that contemporary teaching methods, such as constructivism, 
may have contributed to these findings. In our country, teaching methods and assessments have been aligned 
with international exams like TIMSS and PISA (e.g., MoNE, 2018), which frequently include algorithmic questions 
requiring solid conceptual understanding. Additionally, learning based on semiotics considers knowledge as a 
dynamic process, aligning with modern teaching approaches rather than a static structure to be memorized (e.g., 
Uden et al., 2001). As a result, the gifted students demonstrated greater success in the conceptual test than in 
the algorithmic test, contrary to the prevailing literature (e.g., Duzgtin et al., 2001; Mason et al., 1997; Nakhleh, 
1993; Niaz, 1995; Pickering, 1990; Pushkin, 1998). 

Regarding the second research question of the study, the results indicated that gifted students performed 
the poorest on the graphical test, achieved moderate performance on the algorithmic test, and demonstrated 
the highest performance on the conceptual test. The statistical analysis and comparisons of the students’ perfor- 
mance in all three tests highlight that gifted students’ overall performance on the graphical test was the lowest. 
This finding aligns with previous studies conducted on chemistry topics (Costu, 2007; 2010) and physics topics 
(Costu & Satilmis, 2020; Kurnaz, 2013). Both this study and previous research suggest that both non-gifted and 
gifted students encounter difficulties with graphical questions and struggle to effectively utilize their graphical 
skills, particularly in reading and interpreting graphs. This challenge is not specific to the test items themselves 
but rather stems from students at all levels struggling with various graphical skills, including drawing, reading, 
interpretation, extrapolation, and interpolation, as evidenced by previous studies (Bowen et al., 1999; Bowen 
& Roth, 2005; Costu et al., 2017; Dori & Sason, 2008; Ercan et. al., 2018; Ergul, 2018; Gardner et al., 2021; Glazer, 
2011; Pols et al., 2021; Potgiether et al., 2008; Stefanel, 2019; Tairab & Khalaf Al-Naqbi, 2004). In the context of 
the semiotic construction of scientific knowledge, the poor performance of gifted students on graphical ques- 
tions can be attributed to their struggle in transitioning from one semiotic system to another, as suggested by 
relevant literature (e.g., Duval, 2006; Lemke, 1998; Tang et al., 2011; Tsui & Treagust, 2013; Volkwyn et al., 2020). 
Therefore, it is essential to enhance graphical skills in order to foster a better understanding of physics concepts. 
To achieve this goal, computer-based instruction (e.g., Dori & Sasson, 2008; Sari et al., 2019) and instructional 
strategies (Altun et al., 2011; Harsh & Schmitt-Harsh, 2016; Stefanel, 2019) should be employed, as suggested 
in the literature. 

To address the third research question of the study, comparisons were made between the two types of 
tests. This comparison yielded two major findings. Firstly, the results indicated that the highest-performing 
category among all the categories was the HAHC category, where gifted students were able to effectively 
utilize their conceptual understanding to solve algorithmic questions. This finding suggests that the extent of 
students’ conceptual understanding of physics concepts positively correlates with their algorithmic abilities. 
This result is consistent with certain relevant studies (e.g., Ates & Cataloglu, 2007; Chiu, 2001; Costu, 2007; 2010), 
but inconsistent with others (e.g., Lythcott, 1990; Mason et al., 1997; Nakhleh, 1993; Nurrenbern & Pickering, 
1987; Sawrey, 1990). Secondly, it was observed that there were positive relationships between gifted students’ 
graphical understanding and their algorithmic and conceptual understanding compared to the relationships 
with the other two types. Proficiency in “algorithmic understanding” was associated with success in “graphical 
understanding”. Similarly, strong “conceptual understanding” was linked to good “graphical understanding”. 
Therefore, it is important to enhance both algorithmic and conceptual understanding in order to improve stu- 
dents’ graphical understanding. 


Conclusions and Implications 


The results demonstrated that the gifted students performed significantly worse on the graphical test than 
on the other two tests. Graphs contain both linguistic and symbolic components and can be used to establish 
logical connections between variables or elements within a specific domain of knowledge. The interpretation of 
graphs requires cognitive processes that are distinct from those involved in comprehending linguistic elements. 
The ability to interpret graphs is also a complex skill that requires contextual knowledge and the ability to translate 
unfamiliar graphs into verbal descriptions. Therefore, it can be concluded that both non-gifted and gifted students 
demonstrate lower achievement in questions that involve graphical elements. In the literature, students at all 
academic levels have been found to face difficulties with various aspects of graphical skills, such as drawing, read- 
ing, interpretation, extrapolation, and interpolation, as evidenced by previous studies. In order to enhance these 
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graphical skills, computer-based instruction and various instructional strategies should be used so as to promote 
sound understanding of physics concepts. In the process of learning physics, it is often essential to understand 
and grasp multiple forms of representation. The comprehension of concepts is greatly enhanced by utilizing the 
representations that offer the greatest potential for learning. 

The results also demonstrated that gifted students effectively applied their conceptual understanding to 
solve algorithmic problems and interpret graphs. This finding suggests a positive correlation between students’ 
conceptual comprehension of physics concepts and their proficiency in algorithmic and graphical skills. To enhance 
students’ problem-solving and graphical skills, it is imperative to prioritize the development of their conceptual 
understanding. Conceptual understanding is of utmost importance in science education as it focuses on developing 
a deep understanding of underlying concepts and principles rather than solely memorizing facts or procedures. 
It involves making connections, recognizing patterns, and applying knowledge to new situations. Conceptual 
learning enables students to develop critical thinking skills, problem-solving abilities, and a more comprehensive 
understanding of the subject matter. It promotes higher-order cognitive skills, creativity, and the ability to transfer 
knowledge to real-world contexts. By fostering conceptual learning, students acquire a solid foundation that allows 
them to build upon their knowledge, make meaningful connections, and engage in lifelong learning. 

In this study, the performance of gifted students was examined across different question types, revealing a 
lower achievement in graphical questions. However, the specific graphical skills that posed challenges for these 
students could not be identified, thus indicating a limitation. Future research should aim to comprehensively 
explore the specific graphical skills that gifted students encounter difficulties with. 
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Appendix. 
Example of the three types of questions (namely question 4C, 4A and 4G) 


Sample of the conceptual question type (4C) 


In an environment where air friction is negligible, a body thrown obliquely at an angle 6 with the 
horizontal follows a trajectory as shown in the below figure. 


ascent .” “descent 


Ve 


According to this, about the motion of the object, 
I. The velocity of the shot and the velocity of impact with the ground are equal. 
II. The velocity of the object is zero at T (vertex). 
III. The ascent time of the object is equal to the descent time. 

Which of these statements are true? 


A) Only I B) Only II C) Only III D)I and II E) II and III 


Sample of the algorithmic question type (4A) 
In an environment where friction is negligible, how many meters is the maximum height that an objec 
thrown obliquely at a speed of 50 ?2 m/s at an angle of 45° with the horizontal? (g=10 m/s”) 


A) 180 B) 155 C) 135 D) 125 E) 80 
Sample of the graphical question type (4G) 


Which of the following could be one of the graphs of the horizontal motion of a stone thro 
horizontally from the top of a building? (Air friction is insignificant) 


A) velocity B) velocity C) path 


time 


E) aceleration 
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THE FACTORS THAT INFLUENCE 
THE MOTIVATION TO LEARN 
CHEMISTRY OF UPPER- 
SECONDARY SCHOOL 
STUDENTS IN INDONESIA 


Nur Huda, 
Eli Rohaeti 


Cognitive social theory initiated by Bandura (2001), explains that some 
human learning processes occur in the social sphere. Humans learn by ob- 
serving other people’s attitudes, emotional reactions, and behavior (Bandura, 
1999; Hagger & Hamilton, 2022). This theory developed into a social learn- 
ing theory that is often used to discuss how a person is motivated to learn 
something (Koutroubas & Galanakis, 2022; Schunk & DiBenedetto, 2021). In 
student learning, the social scope is the school, where the learning process 
occurs. So, it is important to create a conducive school environment to in- 
crease student learning motivation. Several ways that can be done include: 
optimizing learning facilities, improving teachers’ learning methods (Ardura 
et al., 2021), and cultivating a sense of belonging to the school (King-Sears & 
Strogilos, 2020; Korpershoek et al., 2020). Nevertheless, in science learning 
practice, it is different. Even though these aspects have been fulfilled, the 
fact is that students’ interest in learning science decreases as they move up 
to grade level (Molnar & Hermann, 2023; Vedder-Weiss & Fortus, 2012). This 
fact suggests that other factors influence the structure of students’ motiva- 
tion when learning science. This has made motivation to learn science an 
interesting topic in education lately. 

Experts state that the structural components of science motivation 
specifically consist of self-efficacy (Schunk & DiBenedetto, 2021; Stewart et 
al., 2020; Stoeckel & Roehrig, 2021), self-determination (Howard et al., 2021; 
Ryan & Deci, 2020), intrinsic motivation, career, and achievement (Bryan et 
al., 2011; Glynn et al., 2011). The components of science motivation are cur- 
rently widely measured through a psychometric instrument called the Sci- 
ence Motivation Questionnaire II (SMQ II). This questionnaire is periodically 
formulated by experts and validated through exploratory factor analysis and 
confirmatory factor analysis in various studies. This questionnaire was not 
only applied to students majoring in science but also to students majoring 
in non-science (Glynn et al., 2011). Other researchers state that the SMQ Il is 
valid and reliable (Dixon & Wendt, 2021; Janstova & Sorgo, 2019; You et al., 
2018). Interestingly, SMQ II can be used widely and adapted to other fields 
of science, such as biology, physics, and chemistry (Glynn et al., 2011). The 
SMQ II was adapted into the Chemistry Motivation Questionnaire II (CMQ II) 
in learning chemistry. 
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Problem Statement 


Chemistry education researchers worldwide compete to use CMQ II as their research instrument. CMQ II has 
been implemented in various countries, such as Greece (Salta & Koulougliotis, 2015), Spain (Ardura & Pérez-Bitrian, 
2018), Turkiye (Cetin-Dindar & Geban, 2015), China (Dong et al., 2020; Zhang & Zhou, 2023a), Brazil (de Souza et 
al., 2022), and other countries. In Indonesia, research of students’ motivation to study chemistry with CMQ II are 
still rare. On average, motivation to study chemistry in Indonesia is measured using other specific instruments 
and cannot be generalized broadly, let alone compared to other countries. Not to mention CMQ II, the use of SMQ 
ll in Indonesia is limited only for dental students (Rahmayanti et al., 2020) and in comparative studies of motiva- 
tion to study science in Korea and Indonesia (Rachmatullah et al., 2018). This fact is a crucial issue to note. If every 
research on students’ chemistry learning motivation is only localized (only in certain areas), then the results will 
not contribute significantly to the development of chemistry education in Indonesia. Given that Indonesia is a 
vast and heterogeneous country. Unfortunately, although CMQ II is widely used in other countries, the fact is that 
using CMQ Il as a research instrument in Indonesia seems less attractive to researchers. Hiding chemistry motiva- 
tion as an affective aspect of students is essential to optimize the effectiveness of chemistry learning in Indonesia. 

CMQ II can extract facts about students’ learning motivation to study chemistry. Several studies state that 
the biggest motivation for students to learn chemistry is to get grades, and few students find the relevance of 
chemistry for their future careers (Austin et al., 2018). Other studies state that motivation to learn chemistry differs 
significantly by gender (Zhang & Zhou, 2023b) and students’ learning experiences (Ardura & Pérez-Bitrian, 2018; 
Salta & Koulougliotis, 2022). 

Many factors cause the learning experience felt by students; it could be a factor of the learning method that 
the teacher applies (Ardura et al., 2021; Dewi et al., 2019), the learning facilities at school, the social environment 
of the school, or the learning system implemented —online or offline (Huang, 2020; Jeffery & Bauer, 2020; Kal- 
man et al., 2020). Some factors affect students’ motivation to learn chemistry because they are related to learning 
anxiety. Bad teachers will increase student anxiety when learning (Stomff, 2014). Online learning systems, such as 
during the COVID-19 pandemic, also caused students to feel anxious, stressed, and depressed (Cervantes-Cardona 
et al., 2022). In addition, practical learning in the laboratory also increases student anxiety (Kurbanoglu & Akim, 
2010). Many studies have mentioned the relevance of anxiety to motivation, but the problem is that none of them 
relates to chemistry motivation. Even in educational studies in other domains, anxiety has been empirically proven 
to strongly influence and correlate with student learning motivation (Al Majali, 2020; Suren & Kandemir, 2020). 
Interestingly, anxiety can increase motivation (Strack et al., 2017) and decrease it (Camacho et al., 2021). This study 
will reveal this correlation in the context of learning chemistry. The Chemistry Anxiety Questionnaire (CAQ) is used 
to measure student anxiety. 

In summary, there were two issues raised in this study. Firstly, the lack of research on students’ chemistry 
learning motivation conducted widely in Indonesia. Second, no specific research discusses the correlation between 
chemistry motivation and student learning anxiety. Those two problems were the reasons why this research needs 
to be conducted. 


Research Aim and Research Questions 


Seeing the complex structural factors of motivation to learn chemistry, using CMQ Il as a research instrument 
is certainly not enough. Motivation is an affective aspect that needs to be explored in depth, so this study does 
not only focus on quantitative data from cross-sectional surveys but also qualitative data from interviews with 
students as a complement. The results of the interviews were useful for enriching the data, so as to reach and 
explain facts that have not been covered by previous researchers. Implementing CMQ II in developing countries 
like Indonesia can produce interesting research data, considering that Indonesia has a unique educational culture 
and is certainly different from other countries. This study aimed to explore the factors influencing upper-secondary 
school students’ motivation to learn chemistry in Indonesia. In line with this aim, the main focus of this research 
was to answer the following questions: 

1. Do differences in gender, class, and anxiety levels influence students’ motivation to learn chemistry? 
2. Are there other factors that influence students’ motivation to learn chemistry? 
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Research Methodology 
General Background 


This research used a quantitative approach with a survey method. The survey design chosen was a cross- 
sectional survey. This research design was implemented to observe without providing intervention to respondents 
in a relatively short period (Lavrakas, 2008) and appropriate for examining participants’ attitudes, beliefs, or opinions 
(Creswell, 2012), so it is appropriate to use to answer questions in this study. Attitudes, beliefs, or opinions can be 
interpreted as an individual's way of thinking about a problem, which in the context of this study is the process 
of learning chemistry. Quantitative data collected through CMQ II were analyzed to detect the five components 
of motivation to learn chemistry based on gender, class and student anxiety level, so statistical calculations were 
needed. Almost all studies with CMQ II were conducted using a cross-sectional survey design, so the research data 
obtained were limited. The survey in this study was combined with interview techniques so the data obtained had 
a deeper meaning. This design was certainly more effective for explaining, especially the possibility of other factors 
influencing students’ motivation to learn chemistry. Qualitative data analysis from interview transcripts enriched 
the perspective of researchers in studying and interpreting the survey data in this study. 

Technically, this study was conducted in two stages; surveys and interviews. The cross-sectional survey using 
CMQ II was completed anonymously by the students within three weeks, while the interviews were conducted 
within two weeks. Overall, this research was conducted over five weeks from January to February 2023. In that 
month, most upper-secondary schools in Indonesia implemented offline learning after being online for a long 
time due to the Covid-19 Pandemic. The survey was conducted directly by distributing paper questionnaires to 
upper-secondary school students. 


Participants 


The survey participants in this study were 1,211 upper-secondary school students in grades 10, 11 and 12 
majoring in Mathematics and Natural Sciences (620 boys and 591 girls) spread across several regions in Indonesia. 
The sample selection is based on the target population of this study, which is upper-secondary school students 
in Java and Sumatra, totaling approximately 1 million. The sample was selected based on the accessibility of the 
researcher to reach the school where the students study. The selection was closely related to school licensing and 
data collection ethics. Another characteristic considered in selecting participants was the same standard of the 
learning process, meaning that all schools where the participants studied were accredited A, had access to comput- 
ers and the internet, and had chemistry teachers who were already undergraduate certified. Based on the survey 
sample calculation with the variance of the population (P) = 50%, accuracy confidence level = 99%, and margin of 
error = 5%, the minimum sample value of 660 students was obtained (Taherdoost, 2017). Thus, the total sample 
in this study is sufficient to represent the target population. 

Allindividuals participating in this study were not subject to intervention or coercion from any party. Students 
have consciously agreed to participate in surveys and interviews without any compensation. All upper-secondary 
school students who participated in this study had received chemistry subjects by the applicable curriculum so 
that participants can objectively answer questionnaires and interviews based on what they feel during learning. 

This study followed the ethical procedures of educational research by considering beneficence, justice, and 
nonmaleficence, especially to the participants. Therefore, the researcher did not conduct data collection in schools 
that did not give permission. In schools where permission was granted, all students could participate in the study 
without distinction of gender, ethnicity, race, or religion. Based on the principle of respecting the rights and dig- 
nity of the individual, all participants in this study were voluntary; thus, all participants involved were ensured to 
be undisturbed and not disadvantaged. All data collected were coded following the principles of anonymity and 
confidentiality. 


Instrument and Procedures 
Students’ chemistry learning motivation was explored through CMQ II (Glynn et al., 2011), while students’ 


anxiety was measured through the CAQ (Chemistry Anxiety Questionnaire) by Megreya et al. (2021), which was 
adapted into Indonesian through five stages, namely: 1) translation, 2) synthesis, 3) back translation, 4) expert com- 
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mittee review, and 5) pre-testing (Beaton et al., 2000). The translation of CMQ Il and CAQ was carried out repeatedly 
by two researchers from the chemistry education study program and one linguist. After that, face validation was 
conducted in a group discussion forum with three chemistry education experts. The Indonesian version of CMQ 
Il consists of 25 question items with five components, namely: intrinsic motivation (IM), career motivation (CM), 
self-determination (SD), self-efficacy (SE), and grade motivation (GM), which can be seen in Table 1. 


Table 1 
Chemistry Motivation Questionnaire II (CMQ Il) 


Items Question Number Loading Factor 
Intrinsic Motivation 
| enjoy learning chemistry Q19 83 
| am curious about discoveries in chemistry Q17 66 
Learning chemistry makes my life more meaningful Q12 82 
Learning chemistry is interesting Q3 19 
The chemistry | learn is relevant to my life Qi 17 
Career Motivation 
| will use chemistry problem-solving skills in my career Q25 84 
My career will involve chemistry Q23 81 
Understanding chemistry will benefit me in my career Q10 83 
Knowing chemistry will give me a career advantage Q13 85 
Learning chemistry will help me get a good job Q7 81 
Self Determination 
| study hard to learn chemistry Q22 83 
| prepare well for chemistry tests and labs Q16 AT 
spend a lot of time learning chemistry Q11 16 
use strategies to learn chemistry well Q6 80 
put enough effort into learning chemistry Q5 15 
Self-Efficacy 
am sure | can understand chemistry Q21 82 
believe | can earn grade ‘A’ in chemistry Q18 69 
believe | can master chemistry knowledge and skills Q15 84 
am confident | will do well on chemistry labs and projects Q14 16 
am confident | will do well on chemistry tests Q9 19 
Grade Motivation 
Scoring high on chemistry tests and labs matters to me Q24 87 
| think about the grade | will get in chemistry Q20 18 
It is important that | get an ‘A’ in chemistry Q8 84 
Getting a good chemistry grade is important to me Q4 19 
| like to do better than other students in chemistry tests Q2 56 


CAQ consists of 9 question items with two components: chemistry learning anxiety (CLA) and chemistry 
evaluation anxiety (CEA), which can be seen in Table 2. Both have answers in the form of a 5-point Likert scale. The 
pre-testing stage of the instrument was carried out on 338 upper-secondary school students. Construct validation 
was done by confirmatory factor analysis (CFA) to confirm whether the questionnaire items were valid for measur- 
ing latent variables from the Indonesian version of CMQ II and CAQ. The software used was Lisrel 8.8. The criteria 
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for the validity of the CFA model were observed based on the goodness of fit indices and the loading factor value 
above .50 (Hair et al., 2010). The reliability of the instrument is calculated using Cronbach's alpha method with 
reliable instrument criteria based on the value of a > .7. 


Table 2 
Chemistry Anxiety Questionnaire (CAQ) 


Hovis Number Loading 
Question Factor 
Chemistry Learning Anxiety 
Starting a new topic in chemistry Q9 12 
Listening to another child in your class explains a chemistry problem Q7 68 
Watching the teacher works out a science experiment on the board Q3 54 
Listening to the teacher talk for a long time in chemistry Q6 15 
Finding out that you are going to have a surprise science quiz when you start your chemistry lesson Q8 53 
Chemistry Evaluation Anxiety 
Taking a chemistry test Q4 19 
Being given chemistry homework with lots of difficult questions that you have to hand in the next day Qs M5 
Thinking about a chemistry test the day before you take it Q2 55 
Having to complete a worksheet in chemistry by yourself Qi 52 


The results of the CFA test showed that all CMQ II items were valid for measuring the components of chemis- 
try motivation and met the goodness-of-fit criteria with a value of y?/df = 1.38, RMSEA = .066, CFI = .99, NFI = .96, 
GFI = .95 and AGFI = .93. CMQ II items also show a loading factor value of > 0.5, so it can be concluded that the 
Indonesian version of CMQ II can be used reliably. Meanwhile, the reliability of the Indonesian version of CMQ II 
also showed a fairly good Cronbach's alpha value, namely a = .969. 

The Indonesian version of the CAQ adaptation also shows sufficient validity and reliability to be used. CAQ 
was constructively reliably used to measure the components of chemistry learning anxiety and chemistry evalua- 
tion anxiety with a value of x?/df = 1.54, RMSEA = .067, CFI = .97, NFl = .96, GFI = .93 and AGFI = .87. All Indonesian 
versions of CAQ items also have a loading factor of > .50. The Cronbach's alpha reliability of the CAQ adaptation is 
a = .803. Thus, the two instruments used in this study were valid and statistically reliable. 

The interviews were conducted using a semi-structured guide arranged systematically by synthesizing simi- 
lar studies. The direction of the questions from the interviews in this study was to dig deeper into what and how 
students feel when learning chemistry at school, as well as to confirm the questions in CMQ II. In terms of validity, 
qualitative instruments are still widely debated (Noble & Smith, 2015). In particular, in this study, the interview 
guidelines were validated through group discussion forums (FGD) with experts (Creswell & Poth, 2016). The expert 
validation interview sheet produced 15 flexible questions that were developed according to the respondents’ an- 
swers. Interview questions in this study were developed based on aspects of chemistry learning motivation so that 
the results of the interviews remained focused on this study’s aims. The interview questions can be seen in Table 3. 


Table 3 
Chemistry Motivation Interview Questions 
Questions Aspects 


Do you think chemistry is an interesting subject or not? 
Intrinsic Motivation 


Is there one personal thing or incident that impressed you with chemistry? 


Do you know what kind of job a chemistry person would take when they graduate from school or college? 


Career Motivation 
Do any of your parents, family or relatives work in chemistry? 
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Questions Aspects 


Do you think chemistry is difficult? 


Self Determination 
What efforts have you made to understand chemistry while at school? 


If suddenly there is an exam, can you answer the chemistry questions (of course, on the topics you have 
learned)? 


Are you confident/unsure about getting a good chemistry exam grade? What is the reason? mere incey 


Do you believe chemistry will lead to success in your future studies or career? 


Do you think your chemistry grades are good or not? 


Is getting a good grade on your report card why you study chemistry? Grade Motivation 


Does the teacher's learning that you currently feel makes you excited to learn chemistry? Can you tell us? 


Do you need learning media (power point, website, application, etc.) when learning chemistry? Does it make 
you more enthusiastic about learning chemistry? 


Extrinsic Motivation (Additional 


Does chemistry lab make you excited to learn chemistry? Or does it make you anxious/worried? Are you Questions) 


anxious/worried? 


What else keeps you interested, excited, and enthusiastic about studying chemistry? 
Data Analysis 


Regression analysis is used to explore the correlation between the independent and dependent variables in 
this study. However, the correlation is predictive or approximate. The dependent variable (y) is predicted through 
the regression equation of two or more predictors according to the model: y= B0 + B1X1 + B2X2 + ... + BnXn + e. This 
study’s independent variables or predictors were gender, class, and student anxiety levels, while the dependent 
variable was the chemistry motivation component. This study includes dichotomous variables (gender and class), so 
they need to be changed into dummy variables by giving codes 0 and 1 (Ghozali, 2021; Tabachnick & Fidell, 2013), 
gender variables are recorded as “female,’ while class variables are coded repeatedly with the k-1 dummy variable 
conversion formula. Of course, the classical assumption must also be fulfilled as a condition for linear regression 
with the ordinary least squares (OLS) approach. The tests performed include multicollinearity, heteroscedasticity, 
normality, and linearity tests. The classical assumption test will be carried out accompanied by multiple linear 
regression tests, and the results will be displayed in a structured manner in this study. 

The feasibility of the regression model is detected through the model feasibility test (F Anova test), regres- 
sion coefficient test (t-test), and the coefficient of determination (value of R-Square or Adjusted R-Square). Based 
on these three tests, it can be known whether there is influence or the magnitude of the proportion of predictor 
influence on the dependent variable. The final stage of the regression analysis is the interpretation of the direc- 
tion and magnitude of the influence of each predictor on the independent variables. All stages of the regression 
analysis were carried out with the help of SPSS version 26 software. 

The final regression model cannot produce a perfect proportion of influence value (R Square). Qualitative 
descriptive analysis of the interview transcripts will be added to complement and confirm the predicted results 
from the regression analysis that has been carried out. That way, the study results will be complete, stronger, and 
able to explore students’ motivation in chemistry more deeply. 


Research Results 


Multiple linear regression was used to analyze the correlation of the dependent variable, namely the five 
components of chemistry motivation (intrinsic motivation, career motivation, self-determination, self-efficacy, and 
grade motivation), with predictors (gender, class, and student anxiety). Classical assumptions must be met in the 
regression analysis, including the multicollinearity analysis of the independent variables. Table 4 is the results of 
the multicollinearity test, which shows the tolerance value and variance inflation faction (VIF). 
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Table 4 


Multicollinearity Test Results 


Independent Variables/Predictors 


Dependent variables CLA CEA xX Xl Female 
Tol. VIF Tol. VIF Tol. VIF Tol. VIF Tol. VIF 
All dependent variables 181 1.280 807 1.239 113 1.402 128 1.373 954 1.048 


Note. Tol. = tolerance; VIP = variance inflation factors 


The table above shows no multicollinearity in the proposed regression model because the tolerance value is 
> .20, and the VIF value is < 10.00 for each predictor. Pearson’s r correlation test in Table 5 also confirms this. Pear- 
son's r correlation test was also used to determine which component best predicts students’ motivation to learn 
chemistry. It could be seen that the predictor with the highest correlation with chemistry motivation was gender 
(female), with a value of .164 <r < .335. In this regression model, the best predictor of the chemistry components 
of motivation was gender. 


Table 5 
Pearson Correlation Test Results on the Dependent Variables 


Correlation coefficient (r) 
Dependent variables 


CLA CEA X Xl Female 
Intrinsic Motivation -.088** .109** .074* .020 .210** 
Career Motivation -.106** 030 A31™ .043 .164** 
Self Determination -.140** .065* .096** -.013 335** 
Self-Efficacy -.065* .083** 045 .019 .202** 
Grade Motivation -.066* -.003 074** .019 .249** 


Note. *p < .05;**p <.01 


The multiple linear regression assumption test continued with the Glejser test to detect heteroscedasticity 
symptoms. The existence of symptoms of heteroscedasticity has implications if the regression model is not accurate 
and efficient so that the proportion of predictor effect cannot be estimated optimally. The Glejser test in Table 6 
proves that the p-value of each predictor for the dependent variable is insignificant (p > .05), meaning that there 
are no symptoms of heteroscedasticity in the proposed regression model. Heteroscedasticity symptoms can also 
be analyzed through irregular scatterplots that do not form a certain pattern. 


Table 6 
Results of the Regression Model Heteroscedasticity Test 


Dependent variables Independent Variable p-value 
CLA CEA xX Xl Female 
Intrinsic Motivation 368 435 054 .093 295 
Career Motivation 390 253 082 674 053 
Self Determination 152 .070 .080 052 113 
Self-Efficacy 625 069 053 .130 055 
Grade Motivation 059 .992 063 077 343 
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The absence of symptoms of heteroscedasticity and normally distributed data give a signal if the linearity of 
the data has also been fulfilled. This classical assumption is important to avoid bias toward estimating the specified 
regression model. Fulfilling the classical assumption implies that the resulting regression model has minimum bias. 
The results of the calculation of the regression model can be seen in Table 7. 


Table 7 
Summary of Regression Models and ANOVA All Predictors of the Chemistry Motivation Component 


Model Summary Anova 
Dependent variables 
R R Adjusted R? F df p 
Intrinsic Motivation 214 075 071 19.633 1210 < .001 
Career Motivation 253 054 060 16.539 1210 < .001 
Self Determination 371 .138 134 38.477 1210 < .001 
Self-Efficacy 237 .056 052 14.292 1210 < .001 
Grade Motivation .260 .067 064 17.445 1210 < .001 
All Variables 326 106 102 28.584 1210 < .001 


It can be seen that the model proposed for all predictors is significant (p < .05) with different R? values. In this 
model, it can be seen that the contribution of the proportion of predictor influence is quite small (.54 < R? < .138). 
The regression model proposed in this study only contributes to the influence of the 5.4% to 13.8% motivational 
component. The rest are other predictors that were not explored in this study. Considering various factors (F, p, 
and adjusted R’ values), the above model is considered as the best and shows the largest R? value compared to 
other model variations. 

In multiple linear regression, the value of b indicates the contribution of each predictor to the model. The value 
of b can be seen in Table 8, and it can be seen that almost all p values are significant (p < .05). Some predictors that 
are not significant (p > .05) will be removed from the equation because they do not affect the dependent variables. 


Table 8 
Predictor Regression Coefficient on the Chemistry Motivation Component 


Coefficient 
Dependent variables Predictors 
b b* SD t p 

Constant 15.273 - 0.428 35.683 001 
CLA -0.094 -0.120 0.025 -3.818 001 
CEA 0.160 0.172 0.029 5.569 001 

Intrinsic Motivation 
X 0.591 0.087 0.223 2.646 .008 
Xl 0.312 0.052 0.190 1.640 101 
Female 1.017 0.184 0.165 6.474 .001 
Constant 15.236 - 0.503 20.279 .001 
CLA -0.093 -0.101 0.029 -3.193 .001 
CEA 0.096 0.088 0.034 2.830 .005 

Career Motivation 
X 1.463 0.184 0.263 5.574 .001 
Xl 0.906 0.132 0.233 4.054 .001 
Female 0.876 0.129 0.194 4.504 .001 
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Coefficient 
Dependent variables Predictors 
b b* SD t p 

Constant 15.629 - 0.439 35.583 001 

CLA -0.114 -0.136 0.025 -4.503 001 

CEA 0.137 0.138 0.030 4.649 001 

Self Determination 

X 0.511 0.071 0.229 2.228 026 

Xl 0.086 0.014 0.195 0.439 661 

Female 1.904 0.307 0.170 11.217 .001 

Constant 16.115 - 0.459 35.207 001 

CLA -0.068 -0.081 0.027 -2.561 011 

CEA 0.125 0.126 0.031 4.041 .001 

Self-Efficacy 

X 0.352 0.049 0.239 1.471 142 

Xl 0.223 0.036 0.204 1.094 274 

Female 1.146 0.185 0.177 6.459 .001 

Constant 17.093 - 0.483 35.392 .001 

CLA -0.020 -0.022 0.028 -0.707 480 

CEA 0.019 0.018 0.032 0.579 562 

Grade Motivation 

X 0.598 0.078 0.252 2.373 018 

Xl 0.375 0.057 0.215 1.749 .081 

Female 1.556 0.237 0.187 8.336 .001 


Based on the results of the regression calculation table above, with consideration of the significance value (p 
< .05), the regression equations are produced as follows: 

IM = 15.273 - 0.094 (CLA) + 0.160 (CEA) + 0.591 (X) + 1.017 (Female) 

CM = 15.236 - 0.093 (CLA) + 0.096 (CEA) + 1.462 (X) + 0.906 (XI) + 0.876 (Female) 

SD = 15.629 - 0.114 (CLA) + 0.137 (CEA) + 0.511 (X) + 1.904 (Female) 

SE = 16.115 - 0.068 (CLA) + 0.125 (CEA) + 1.146 (Female) 

GM = 17.093 - 0.598 (X) + 1.556 (Female) 


This equation is used to answer the first question in this study. Based on the analysis of multiple regression 
tests, it can be concluded that not all predictors (gender, class, and level of anxiety) significantly affect aspects of 
motivation to learn chemistry. The components of intrinsic motivation (IM), self-determination (SD), and self-efficacy 
(SE) were influenced by gender and students’ anxiety levels. In comparison, career motivation (CM) is influenced 
by all predictors, while the aspect of class motivation (GM) is only influenced by students’ gender. Based on the 
regression equation, it can be seen that gender is the most influential predictor of the motivational component. At 
the same time, anxiety affects all components of chemistry motivation except classroom motivation. Meanwhile, 
class differences did not affect the motivational component other than career motivation. 

The maximum value of the proportion of the influence of the proposed regression model is 13.8%. The rest 
(86.2%) were other predictors not included in the proposed regression model. The value of the resulting propor- 
tion is quite small, but this estimation must be accepted as a statistical fact considering that the construction of 
motivation to learn chemistry is quite complex. Therefore, additional research data is needed. In this case, it is the 
analysis of the results of interviews with respondents. A summary of the interview transcript coding was compiled 
and can be seen in Table 9 for easier understanding. Coding is done by grouping certain themes to extract other 
factors influencing motivation to learn chemistry based on the students’ perspectives. 
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Coding of Other Factors That Influence Chemistry Motivation 


Extracted aspects 


Factors 
1 2 3 
Learning experience Teachers’ learning methods and Laboratory experiment The learning media used by the 
models teacher 
Chemistry concept confusing chemistry concepts Chemistry lessons that contain calcula- | Chemistry concepts get more 
tions difficult as the class progresses 
Learning environment Friendship and class environment School facility Family career in chemistry 
digital literacy Get to know chemistry via the Learn chemistry via the internet Doing chemistry assignments 
internet via the internet 
Students’ anxiety Teachers who are not good at Sudden chemistry test Laboratory experiment 


teaching 


There were five factors extracted from the interview transcript analysis, including 1) students’ learning experi- 
ences, 2) chemistry concepts, 3) students’ environment, 4) students’ digital literacy level, and 5) students’ anxiety. 

Concerning the students’ learning experiences, the aspect that appears most often is teacher learning. All 
interview respondents agreed that the teacher is the most influential factor in their motivation to learn chemistry. A 
good chemistry teacher will increase students’ motivation to learn chemistry. In addition, the practical aspect cannot 
be forgotten. Students’ motivation to learn chemistry is closely related to learning in the laboratory (practicum). 
Besides not being boring, learning with experiments in the laboratory is more meaningful and more manageable 
for students to understand. 

Factors that influence motivation to learn chemistry do not arise only from the outside. The concept of 
chemistry is an internal factor influencing students’ motivation to learn chemistry. Chemistry content is unique 
and specific, which includes concepts and calculations. Not all students are interested in chemistry because of the 
characteristics of chemistry concepts. 

In line with the cognitive social theory, the environment is one of the factors that influence motivation to 
learn chemistry based on the results of the interviews. The environment in this context is related to the friendship 
conditions of students, school facilities, and family careers in chemistry. Students with relatives who work in the 
chemistry field are indicated to be more motivated to learn chemistry; this is related to career motivation. 

Students’ motivation to learn chemistry is also related to digital literacy. The accessibility of students to the 
internet allows them to get to know and learn more about chemistry. Students’ proximity to the internet indirectly 
motivates them to learn chemistry more deeply. 

The last factor synthesized from the interview results is the anxiety factor. The chemistry anxiety questionnaire 
(CAQ) and multiple linear regression tests in this study have predicted the anxiety factor. The students’ anxiety was 
evident from the analysis of the interview transcripts. Another aspect of anxiety that needs to be considered as a 
motivating factor is the nature of the teacher when teaching. Student motivation tends to decrease when they are 
taught by teachers who are not communicative. In a sense, teachers are too rigid when teaching, often give sud- 
den tests and get angry if students do not understand the chemistry concepts. In addition, in terms of practicum 
anxiety, male students tend to be more motivated to learn chemistry in laboratory practice than female students. 


Discussion 


This study aimed to explore the factors influencing upper-secondary school students’ chemistry learning 
motivation through multiple regression analysis and analysis of interview transcripts. In the multiple regression 
analysis, the predictors determined were gender (female), class (X and XI), and chemistry anxiety (chemistry learn- 
ing anxiety and chemistry evaluation anxiety). The introduction explains that these three predictors are likely to 
influence students’ motivation to learn chemistry. This study provides strong evidence that not all predictors have 
a significant effect, even though they are statistically correlated. 
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Concerning the correlation between the independent and dependent variables, measurements were carried 
out with the Pearson correlation coefficient (rn), which can be seen in Table 4. The data shows two exciting things: 
the level of correlation between variables included in the low category and the predictor of chemistry learning 
anxiety (CLA) that is negatively correlated. This low level of correlation is also confirmed by the results of R? in 
Table 7. Gender in this study is the predictor with the highest correlation; this confirms other studies, such as those 
conducted by De Souza et al. (2022) and Zhang and Zhou (2023). 

Previous studies have shown that gender has a relatively small or medium effect (Glynn et al., 2011; Zhang 
& Zhou, 2023); this study also says so. Gender only had a slightly statistically significant effect (.164 <r < .335). 
The most considerable correlation of gender predictors is on the self-determination variable, which also confirms 
previous science research (Glynn et al., 2009, 2011). Correlation with a positive value implies that female students 
are more motivated to learn chemistry than male students. Gender predictors are statistically significantly differ- 
ent; the results of this study are the same as in previous studies (Salta & Koulougliotis, 2022; Zhang & Zhou, 2023). 

The analysis results were also supported by the fact that upper-secondary school students in Indonesia who 
went on to major in chemistry were predominantly female. Even in some universities in Indonesia, male chemistry 
majors make up at most 20% of students. In addition, academic research in the UK also states that women prefer 
chemistry at the upper-secondary school level (Crossdale et al., 2022). Gender disparity research in chemistry 
learning was also conducted in central Florida by Semerzier (2021). The research implies that female students 
(black and Latina) dominate the university community. This gender factor ultimately impacts all career stages in 
the world of work in chemistry. 

Another predictor that was measured in this study was class differences (X, Xl, and XIl). The low coefficient 
value (-.013 < r<.131) implies that class differences did not significantly influence students’ motivation to learn 
chemistry. In previous studies, it has been revealed that academic majors have low and moderate correlations (Salta 
& Koulougliotis, 2022). In line with this study, at the upper-secondary school level, class differences also did not 
significantly affect motivation to learn chemistry. An analysis of the interview results confirmed these results. The 
interview transcripts show no indication that class differences influence motivation. However, two respondents 
said that chemistry concepts became more difficult as the class level increased—this interview results are in line 
with the study by Molnar and Hermann (2023) and Vedder-Weiss and Fortus (2012). So, there is a tendency that 
the higher the class level of students, the lower their motivation to learn chemistry. 

The decline in motivation to learn chemistry made upper-secondary school students not optimally do the 
national or end-of-school exams when class XII. So, it is unsurprising that many upper-secondary school chemistry 
exam scores in Indonesia are in the low category (Ferdhiana et al., 2017). Not to mention, Indonesia has many areas 
classified as rural and needs help accessing educational facilities. Thus, the chemistry exam results of students in 
rural areas are consistently lower than those of students in urban areas (Adlim et al., 2014). This socio-demographic 
factor causes the motivation to learn chemistry among upper-secondary school students in Indonesia to be quite 
heterogeneous. 

Chemistry anxiety, which was appointed as a predictor in this study, also did not have a statistically signifi- 
cant effect on motivation to learn chemistry (-.140 < r < .109). However, an interesting finding that needs to be 
considered is the low correlation of the predictor of anxiety level, which is hostile (-.140 < r < -.065) on students’ 
motivation to learn chemistry. The implication is that there is an inverse correlation between the two variables. The 
higher the students’ anxiety, the lower their motivation to learn chemistry. This interpretation is also confirmed 
by the negative regression coefficient b in Table 6. Analysis of the interview results also reinforces this. An anxiety 
factor is extracted and influences students’ motivation to learn chemistry. The causes of chemistry learning anxiety 
revealed by respondents were anxiety when taught by uncommunicative teachers and when experimenting in 
the laboratory. This anxiety is, in fact, also closely related to the learning experience in class, as revealed in another 
CMQ II study by Vedder-Weiss and Fortus (2013) and Zhou et al. (2019) 

The teacher can reduce anxiety in learning chemistry. One way is collaborative active learning (Guo et al., 
2022). An active learning environment can increase students’ interest and competence in chemistry (Hendrikson, 
2021). Passive and dogmatic students tend to be anxious and worried when receiving chemistry learning in class. 

Based on the five regression equations extracted from statistical calculations, insignificant effects are often 
found in predictors of class differences. Meanwhile, the predictor that always appears in the regression equation is 
gender difference (female). This finding contrasts the results of a study by Salta and Koulougliotis (2022) which said 
that gender was not included in the regression equation that was formed. It can be said that gender difference in 
this study is the predictor that has the highest predictive power of students’ learning motivation to learn chemistry 


625 


https://doi.org/10.33225/jbse/23.22.615 LQ 


Journal of Baltic Science Education, Vol. 22, No. 4, 2023 


THE FACTORS THAT INFLUENCE THE MOTIVATION TO LEARN CHEMISTRY OF UPPER- ISSN 1648-3898 /Print/ 
SECONDARY SCHOOL STUDENTS IN INDONESIA 
(PP.615-630) ISSN 2538-7 1 38 /Online/ 


(with a value of b, 0.876 < 6 < 1.904). In terms of grade motivation and self-determination, the predictive value 
of gender is the largest compared to the others. These statistical results prove that gender predicts motivation to 
learn chemistry that must be maintained in the context of other motivation to learn chemistry studies. If you look 
at the interview transcript, then there is a gender difference which is quite striking. For example, answering the 
question, “What made you interested in chemistry?” 

Male respondents tended to answer in the context of experiments in the laboratory, as stated in the opinion 
of participant ID5 below. 


“| prefer studying chemistry while doing experiments in the laboratory. Practicing in the laboratory is more interesting than 
learning theory in class. With practicum, | understand the material presented more quickly. Apart from that, the practicum 
is fun and does not make you bored,” 


The ID5 participant's opinion is correct. Learning in the laboratory does increase motivation to learn chemistry 
(Yunita, 2017). The problem is that many schools in Indonesia need lab facilities. However, virtual labs can overcome 
this (Sasmito & Sekarsari, 2022). Meanwhile, female respondents tended to answer in the context of classroom 
learning and interesting chemistry content, as stated in the opinion of Participant ID1. 


“Learning chemistry is fun because the teacher is good at teaching it. | like to learn new things like elements and chemistry 
reactions. | think chemistry is easy enough for me to understand.” 


Based on these two opinions, it can be concluded that gender remains one of the predictors that significantly 
affects motivation to learn chemistry, even though the correlation is low. Teachers with suitable learning meth- 
ods will undoubtedly increase students’ learning motivation (Salta & Koulougliotis, 2015). Thus, teachers need to 
integrate modern learning media such as augmented reality, virtual reality, gamification, and 3D visualization to 
increase the attractiveness of chemistry lessons in the eyes of upper-secondary school students. 

Regarding the negative Pearson correlation test results, predictors of anxiety levels should not be underesti- 
mated either. This study provides sufficient evidence to state that students’ anxiety significantly affects motivation. 
The higher the students’ anxiety, the lower their motivation to learn chemistry. This was confirmed by the opinion 
of participants ID3. 


“One thing | was worried about when studying chemistry was teachers who were not communicative. Sometimes, teachers 
like that give explanations that are difficult to understand and often give sudden tests without notification.” 


The influence of the teacher on the motivation to learn chemistry certainly also applies vice versa. In Indonesia, 
there are still many teachers who are temperamental when teaching. So that there was no activeness and freedom 
of learning from students. Students tend to be afraid and passive when learning in class. A temperamental teacher 
will only increase student anxiety. Such anxiety certainly has a negative impact on learning chemistry (Chhetri et 
al., 2022). 

Returning to the fact that motivation is an affective aspect of humans, it is unsurprising that many factors 
influence it. The regression model proposed in this study only contributes 13.8% of the total proportion of chemistry 
motivation factors; other factors have also been extracted from the interview transcripts. Of course, other factors 
need to be studied and explored more deeply. Thus, studies on students’ motivation to learn chemistry are getting 
more complete and positively contributing to the future of chemistry learning. 


Conclusions and Implications 


This study tries to reveal whether differences in gender, class, and anxiety levels influence students’ motiva- 
tion to learn chemistry. Aspects of chemistry motivation as the dependent variable in this study include intrinsic 
motivation (IM), career motivation (CM), self-determination (SD), self-efficacy (SE), and grade motivation (GM). 
Through multiple regression analysis and interview transcript analysis, it can be concluded that: 

Gender differences affect motivation to learn chemistry, although the correlation is not large. Female students 
have a higher incentive to learn chemistry than male students. The regression analysis found that gender was still 
consistent in providing the proportion of influence on each aspect of chemistry motivation. Analysis of the results 
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of the interviews also said so. The perspectives of male and female students are opposite in answering the same 
questions. Thus, future studies on students’ motivation to learn chemistry are expected to consider gender aspects 
as one of the predictors. 

The class difference has a smaller proportion of influence than other predictors. Thus, this study has provided 
sufficient evidence that class differences are not urgent to be used as a motivational factor for learning chemistry. 
In addition, low correlation does not have important implications for a study, and chemistry learning. 

The predictor to consider is students’ anxiety. This study proves that anxiety has a significant and statistically 
negatively correlated effect. It would be interesting to explore the level of students’ anxiety as a predictor of mo- 
tivation to learn chemistry in other contexts. 

As for the implications for future studies, CMQ Il as an investigative instrument of motivation to learn chem- 
istry will be more complete if combined with qualitative research methods. This study will be more accurate if it 
uses sequential explanatory mixed methods so that the effect of the predictors investigated can be explored more 
deeply andit can also extract other predictors. This study also extracted other factors influencing motivation to learn 
chemistry from the interview transcript analysis, including learning experience, chemistry content, environmental, 
and digital literacy factors. Thus, other researchers in the future may use these factors as predictors of motivation 
to learn chemistry in different study contexts. 

For the implications for learning chemistry in upper-secondary school, the teacher figure is still the most deci- 
sive aspect in shaping students’ motivation to learn chemistry. Thus, teachers must consider methods, models, and 
learning media. It is also essential to pay attention to the intensity of learning in the laboratory when delivering 
chemistry material, considering that chemistry contains materials that need to be studied based on experiments 
in the laboratory. Other things teachers need to consider when teaching chemistry are optimizing digital literacy 
in chemistry, creating a comfortable school environment for learning chemistry, and providing psychological 
support for students. 
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Introduction 


Assessment is an integral part of the educational process, which is used 
to observe and analyze the achievement level of students regarding their ex- 
pected learning outcomes. This is applied to determine whether the learning 
process is in line with the expected objectives or still requires development 
and improvement. The nature of the assessment is to look for information 
that has been obtained and then provide quality feedback to improve fu- 
ture performance. While both processes involve collecting data about work 
performance or product, what is done with those data in each process is 
substantially different and evokes very different mindsets. Assessment for 
learning programs is based on the premise that formative assessment guides 
and facilitates learners in reaching their full potential (Ewim & Opateye, 2021). 
In this condition, assessment plays an important role in achieving an active 
collaborative learning environment, providing formative and summative data 
for both teachers and students. 

Some of the findings of the application of assessment in the learning 
process have been carried out including the application of integrated technol- 
ogy in formative assessment to improve students’ conceptual knowledge skills 
and motivation in chemical equilibrium material (Hagos & Andargie, 2022). 
Although it can increase knowledge and motivation, this formative assessment 
has not been able to reveal students’ scientific performance because the process 
of giving feedback takes a long time. In the applied curriculum, learning assess- 
ment is used to measure the level of cognitive abilities and all other aspects of 
skills, such as thinking skills and scientific performance skills. Some students 
improve their metacognitive skills by observing learning processes. Students 
improve their problem-solving thinking, strategies, and solution creation 
through discussion and trial-and-error. Furthermore, students enjoyed learn- 
ing and communicated well in groups by posing questions to solve problems 
(Anwari et al., 2015). Research centered on developing scientific performance 
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Abstract. /n the virtual laboratory learning 
process, students’ scientific abilities in 
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to explore. This study aims to develop 
classroom-based authentic assessment 
instruments through virtual laboratory 
learning in chemistry to see an increase 
in students’ scientific performance. The 
research was conducted at a public high 
school in Langsa City with a sample 

of 11th-grade students Academic Year 
2021/2022, the utilized subjects being 

a total of 118 students taken based on 
purposive sampling. The research data 
are needs analysis through interviews 
with chemistry teachers who have 
implemented technology-based learning, 
analyzing the feasibility of instruments 
through expert validation questionnaires 
before the learning process, and scientific 
performance through learning observation 
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in improvement after the authentic 
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that authentic assessment is feasible 

to use an increase in students’ scientific 
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the students’ scientific performance was 
a high category and there is a significant 
difference in improvement. The use of 
authentic assessment through virtual 
laboratories is expected to provide active 
learning solutions to improve student 
learning outcomes. 
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has been carried out by (Cruz Neri et al., 2021; Price et al., 2010) but this study has limitations in comparing scientific 
performance at the beginning of learning and the end of learning. Assessment of the overall learning process has 
been conducted but only at the basic institutional level (Ibarra-Saiz et al., 2021). 

The condition that is often found today is that the assessment of learning conducted in class only emphasizes 
mastery of the concepts taught by written objective and subjective tests. The tests carried out only measure mastery 
of the material and evaluate low-level cognitive domains (Millah et al., 2021). Survey findings indicate that to deter- 
mine teachers’ needs for authentic assessment of Grade 8 writing skills, teachers require various types of evaluation, 
including writing samples, process writing, portfolio, performance assessment, journal, and projects/exhibitions 
(Refnaldi et al., 2017; Ulhasanah et al., 2020). 

Research limitations explain that authentic assessment instruments display complicated task details while teach- 
ers need simple assessment rubrics. The results of a complicated rubric also show the reasons teachers are reluctant 
to carry out learning activities that focus on developing students’ attitudes and scientific performance skills (Azzahra 
& Simatupang, 2021). An assessment that can assess all aspects of student learning outcomes (which are included 
in the cognitive, affective, and psychomotor aspects) both as a result of learning processes, as well as in the form of 
changes and developments in activities, and the acquisition of learning during the learning process takes place in 
the classroom and outside the classroom with authentic assessment (Hanifah & lrambona, 2019; Sukma et al., 2022). 

Authentic assessment requires students to work in certain concrete and meaningful situations (Miller & Kon- 
stantinou, 2022; Nguyen & Phan, 2020). According to this authentic assessment, not much is applied appropriately in 
learning. More than half of the teachers polled stated that the assessments were not properly designed, implemented, 
or processed. The picture remains that the most extensive implementation of learning assessments in schools is still 
the dominant type of assessment tool used, such as writing tests (Hanifah & lrambona, 2019; Karunanayaka & Naidu, 
2021). The results of the study also explain that most teachers have not mastered authentic assessment instruments 
and the instruments are not suitable for expressing student performance skills. 

Based on previous research, it is necessary to design authentic assessments that have been validated so that 
teachers in schools can apply them. In addition, it is also necessary to apply learning activities that can express stu- 
dent performance skills by adding technology to the learning process. Laboratory-based learning is a way to reveal 
student performance skills. The role of laboratory is important in building students’ skills and knowledge (Agustian, 
2019; Agustina & Churiyah, 2019; Seery et al., 2019). There have been many uses of virtual laboratories (Ali et al., 
2022; Tatli & Ayas, 2010; Yildrim, 2020), each of which applies to science learning. Design and development of virtual 
laboratories oriented to education can improve motivation to perform the various practices in the laboratory (Billah 
&Widiyatmoko, 2018; Evstatiev et al., 2022; Sahroni et al., 2020; Tuystiz, 2010). The results of the literature review show 
that the virtual laboratory is an interactive learning solution that can make it easier for students to understand abstract 
concepts to become more real, but no one has yet revealed an increase in scientific performance. 

The aim of this research is to find how teachers need material for implementing virtual laboratories, how teachers 
apply authentic assessments in schools and what obstacles teachers face during the assessment process, and how 
they influence students’ scientific performance improvement when virtual laboratory learning is applied. The aim is 
to find out the materials needed that apply virtual laboratories, find out authentic instruments that are suitable for 
use, and increase student scientific performance. 


Research Methodology 
General Background 


In this research, the background includes an analysis of material needs that allows a virtual laboratory to be 
held as a basis for implementing authentic assessments, the feasibility analysis of several validators, and data analy- 
sis of scientific performance. The contribution of this research does not only come from virtual experiments and 
group discussions but also from evaluating the application of authentic assessments with more possible obstacles 
encountered in the field to suggest future research needs. The research data are needs analysis through interviews 
with chemistry teachers who have implemented technology-based learning, analyzing the feasibility of instruments 
through expert validation questionnaires before the learning process, and scientific performance through learning 
observation sheets during the learning process. Qualitative data in the form of descriptions of student responses 
are given at the end of the lesson to see the achievement of authentic assessment implementation. 
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Participants 


Participants were from 3 public schools comprising 3 chemistry teachers and 11‘*-grade students academic 
year 2021/2022 totaling 118 students from the best schools in Langsa City, Aceh Province. The average age of stu- 
dents participating in this study was 16 years old and there were 55 males and 63 females. The age of the teacher 
participants ranged from 26 years to 40 years with similar background qualifications. Gender was not considered 
for teachers as it did not feature in the research questions for this study. The study was conducted for four weeks 
during the teaching process using PhET interaction simulation applications on https://phet.colorado.edu/ in topic 
chemistry on https://phet.colorado.edu/en/simulations/filter?subjects=chemistry&type=html, prototype. The expert 
validator participants came from 1 well-known university in Aceh Province consisting of 3 chemistry lecturers aged 
30-50 years who have experience as validators of chemistry education research. 


Instrument and Procedures 


The instrument for analyzing the needs of chemistry virtual laboratory learning materials is in the form of a 
questionnaire consisting of indicators of conceptual understanding, procedural complexity, and lack of sufficient 
resources. The needs analysis questionnaire sheet is given at the time before designing an authentic assessment. 
The results of the needs analysis data become a reference for materials that will be used as teaching materials 
using a virtual laboratory. After the needs analysis stage, the researcher continued the formulation stage of the 
authentic assessment instrument grid for media expert and material expert assessment tools as well as product 
practicality instruments at the expert validation stage. The details of the categories in the media expert validation 
instrument, materials, and product practicality tests can be seen in Table 1. 


Table 1 
Details of Research Instrument Categories 


Subject Categories of validity and practicality 
Media expert Relevance, Completeness, Accuracy, and Clarity 
Material expert Completeness, Legibility, Clarity, Appropriateness, and Accuracy 
Practitioners Accuracy (Effectiveness and Efficiency), Practicality, Clarity, and Completeness 


Using observation sheets and activity assessment rubrics, an authentic assessment instrument model is used 
to measure student activity in understanding scientific performance. This assessment is directly conducted during 
the learning process, from start to finish. Authentic student assessment is carried out in the science laboratory room 
for 2 hours a week and lasts 8 weeks. Scientific performance data were obtained through observation sheets, by 
setting the highest and lowest values of 5 and 1, respectively, with the indicator and scoring rubric shown in Table 2. 


Table 2 
Rubric for Assessing Student Scientific Performance 


Indicator Rated aspect Rubric Score 
Formulating Proper use of laboratory Students use safety equipment completely and correctly 5 
problems Safely equipment Students use safety equipment correctly and incomplete 3 

Students use safety equipment incorrectly and incompletely 1 
Preparation of practical tools © Students prepare tools and practicum materials completely and correctly 5 
andimatetials Students prepare tools and practicum materials correctly and incomplete 3 
Students prepare tools and practicum materials that are not correct and 1 


incomplete 
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Indicator Rated aspect Rubric Score 
Formulating predic- Planning of work procedures _— Students plan work procedures completely and correctly i] 
uns Students plan work procedures correctly and incomplete 3 
Students plan work procedures incorrectly and incompletely 1 
Formulating Filling in the work practicum Students fill in the observation table completely and correctly 5 
experimental vari- observation table using PhET — ; ; 
ables (responses, simulation on https://phet. Students fill in the observation table correctly and incompletely 3 
a and coloraaccedl Students fill in the observation table incorrectly and incompletely 1 
Formulating op- Filling in the results of work Students fill in the observations completely and correctly 5 
ail FER merotiee Students fill in the observations correctly and incomplete 3 
variables phet.colorado.edu/ Students fill in the results of observations that are not true and incomplete 1 
Communicating Determination of observation Students determine the results of observations completely and correctly 5 
raeuldoacs ia bare using tonne and Students determine the results of observations are correct and incomplete 3 
graphs Students determine the results of observations are not true and incomplete 1 
Analyzing experi- Analysis of experimental Students analyze the observations completely and correctly 5 
ial ale ceutesilte Students analyze the results of observations that are correct and incomplete 3 
Students analyze the results of observations that are not true and incomplete 1 
Making conclusions — Conclusion of the final result © Students conclude the observations completely and correctly 5 
pin abeatyaton Students conclude that the observations are correct and incomplete 3 
Students conclude that the observations are incorrect and incomplete 1 


Student responses to authentic assessments use response questionnaires. Student response data were ob- 
tained using a questionnaire consisting of indicators of accuracy (effectiveness and efficiency), practicality, clarity, 
and completeness. Student response questionnaire sheets are given at the end of learning to see the effectiveness 
of implementing authentic assessments in improving student scientific performance. In this condition, the data 
obtained include virtual laboratory material data, validation data, student scientific performance assessment data, 
and student response questionnaire data. 


Data Analysis 


The need for virtual laboratory learning material, authentic instrument feasibility questionnaire data were 
analyzed by calculating the percentage. In this condition, validity is the extent to which the measurement function 
of a measuring instrument is carried out, based on its suitability and accuracy (Feereira, 2016; Zacharis, 2010). The 
increase in students’ scientific performance is observed from the pre-test and post-test scores using the N-Gain 
formula. In this condition, the calculation formula adopted the opinion of (Guntara, 2021), where data processing 
and analysis were carried out by subtracting the final scientific score from the initial (V5_ ng )> This was divided 
by the ideal scientific work minus the initial (Nyy 45 Na) and multiplied by 100%. Basic descriptive statistics (N, 
M, SD) of the numerical variables were determined. An independent sample t-test was used to analyze the differ- 
ence in the pre-test and post-test of the students’ scientific performance. In addition, the significance level of the 


independent sample t-test in this research was set to p = 0.5. 
Research Results 
Quantitative Results 
According to the interviews with 3 chemistry teachers in Table 3, acid-base material represented 84.3% of the 


material needed for student assessment. This is due to the frequent evaluation of the acid-base topic at national 
exams. 
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Table 3 


Data Analysis of Authentic Assessment Needs 


Factors (%) 


THE FEASIBILITY AUTHENTIC ASSESSMENT INSTRUMENT THROUGH VIRTUAL LABORATORY 
LEARNING AND ITS EFFECT ON INCREASING STUDENTS’ 


SCIENTIFIC PERFORMANCE 


(PP.63 1-640) 


Learning materials CU PC SR Average level 
Respondents of need (%) 
1 2 3 1 2 3 1 2 3 
Acid-base 80.0 80.0 80.0 83.0 87.0 85.0 88.0 88.0 88.0 84.3 
Substance purification 73.0 77.0 75.0 72.0 80.0 76.0 78.0 78.0 78.0 76.0 
Chemical reactions 80.0 86.0 83.0 80.0 74.0 78.0 78.0 82.0 80.0 80.0 
Solution stoichiometry 82.0 88.0 85.0 77.0 83.0 80.0 86.0 80.0 83.0 82.7 
Thermochemistry 77.0 77.0 77.0 81.0 75.0 78.0 70.0 76.0 73.0 76.0 


Note: CU = Conceptual Understanding, PC = Procedural Complexity, SR = lack of Sufficient Resources 


The feasibility results of authentic assessment instrument validation based on Figure 1 show that media experts 
have the highest validation percentage, namely the relevance indicator with an average of 87%, subject matter 


experts’ the completeness indicator is 87% and the practicality indicator completeness is 89%. 


Figure 1 


The Feasibility of Expert Validation on Authentic Assessment Instruments Through Virtual Laboratory Learning 
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Each validation in Table 4 showed that the authentic assessment instrument was very valid, with an average 
percentage value of 84%. These results were used to improve performance assessment instruments and portfolios, 
regarding the experts’ judgments. They were also used to observe students’ scientific performance, with improve- 
ments being performed according to the validator's suggestions. Furthermore, reliability specifically represented 
the quantity of error variance with various estimated values, for instance, test-retest, generalizable theory, and 


internal consistency. 


https://doi.org/10.33225/jbse/23.22.631 


635 


<P 


Journal of Baltic Science Education, Vol. 22, No. 4, 2023 


THE FEASIBILITY AUTHENTIC ASSESSMENT INSTRUMENT THROUGH VIRTUAL LABORATORY ISSN 1648-3898 /Print/ 
LEARNING AND ITS EFFECT ON INCREASING STUDENTS’ SCIENTIFIC PERFORMANCE 

(PP.63 1-640) ISSN 2538-7 138 /ontines 
Table 4 


Details of Research Instrument Categories 


Subjects Indicators Score (%) Interpretation 
Assessment expert Relevance, Completeness, Accuracy, and Clarity 88.0 Very valid 
Linguist Completeness, Legibility, Clarity, Appropriateness, and 78.0 Valid 

Accuracy 
Practitioner/elementary school Accuracy (Effectiveness and Efficiency), Practicality, Clarity, 85.0 Very valid 
teacher and Completeness 

Average 84.0 Very valid 


The small-scale trial data conducted on 11*-grade students in Table 5 also showed different results in each 
indicator. From the small-scale test, N-gain the average value increase in the students’ scientific performance was 
0.7 and observed in the high category. 


Table 5 
N-gain of Gender Difference in Students’ Scientific Performance 


Participant Gender Pre-test Post-test Gain N-gain Average Category 
Female 3.5 46 1.5 0.73 
Experiment class 0.7 High 
Male 3.2 44 18 0.67 


Data on students’ scientific performance scores in Figure 2 shows the difference in pre-test and post-test 
scores, where the inference indicator has the highest score, namely 86 compared to other indicators. 


Figure 2 
Small-Scale Trial of Student Scientific Performance 
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Table 6 shows the independent sample t-test is significant difference in increasing students’ scientific perfor- 
mance in the groups in the pre-test for females and males (t = 0.06 and p = .05). However, in the post-test, there 
was a significant difference (t = -1.7, p < .05) between females and males. 
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Table 6 


t-statistic of Gender Difference in Students’ Scientific Performance 


Pre-test Post-test 
Participant Gender 
N M SD t p N M SD t p 
Female 55 3.5 0.89 55 8646 1.24 
Experimental class 0.06 .05 17 < 05 
Male 63 3.2 0.78 63 4.4 1.12 


Qualitative Results 


At the end of the study, the results of student responses to the application of authentic assessment through 
virtual laboratory learning towards improving scientific performance gave the most answers on practicality and 
completeness. The student responses are then described in the form of a comment including: 

Student A: _ | think the authentic assessment is useful. It helps to improve my performance skills when we have a portfolio of 

assignments and step-by-step projects for several weeks. 

Student B: More time to think up ideas for topics about chemistry; more time to write and fix errors; get more information for 

the topic of acid-base material, after step by step, | corrected my writing even though there are still some mistakes. 

Student C: Authentic assessment focuses more on the activities that | do and | become more aware of what | lack in under- 

standing the material. 


Discussion 


In the development of authentic assessment, an instrument is considered appropriate for use in high school, 
because there is an evaluation that can measure student performance in realistic tasks, relevant situations, or prob- 
lems that are useful and meaningful. This shows that the use of authentic assessment can measure students’ abilities 
during virtual practicum activities. Our findings show that acid-base learning is a much-needed lesson using a virtual 
laboratory. Virtual laboratories can display real theories and experiments without direct application in the laboratory 
room. The advantage is that teachers can save time in giving material to students. Virtual learning by helping virtual 
laboratories that can make learning interesting to increase student motivation in enjoying the learning being carried 
out (Estriegana, 2019). 

After implementing the acid-base virtual laboratory, this study also revealed that authentic instruments are very 
feasible to use with the highest completeness indicators. The completeness of authentic assessment instruments is 
marked by indicators for each assessment, namely scientific performance and project-based rubrics. This finding agrees 
that performance assessment and project-based rubrics, as well as concept maps, need to be present in authentic as- 
sessments, to observe students’ cognitive and psychomotor aspects (Salirawati et al., 2021). The performance appraisal 
category has response development and product evaluation (Said, 2020). This shows that performance appraisal and 
process-focused assessment help in teaching and learning activities in the classroom. 

Furthermore, the scientific performance data shown in Table 5 of 0.71 reveal that there is an increase in student 
performance after implementing virtual laboratory learning. As expected, student performance assessment allows 
students to show their abilities, because they can distinguish between methods of conducting real and virtual 
practicums. This assessment is essentially carried out to equip students with real abilities because individual life is a 
process full of creating and solving problems to produce new problems. Under these circumstances, performance 
and portfolio assessments are critical to tracking progress toward those goals. 

The highest student performance assessment score is also found in the conclusion indicator, with an N-gain of 
0.78. Based on these results, the authentic assessment instrument developed can measure scientific performance 
from the highest to the lowest ability. This is due to the ability of performance appraisal to assess students’ knowl- 
edge, attitudes, and skills. In agreement with earlier research that performance appraisal has future implications for 
students regardless of level, developing a foundation for problem-solving, independent learning, and constructive 
collaboration (Barber et al., 2015). Asa task, this learning tool also requires students to show some meaningful realistic 
performances, related to the application of knowledge, understanding, and skills (Ernawati & Sujatmika, 2021; Price 
et al., 2010; Sulistyo et al., 2020). 

This study found the fact that performance appraisal allows students to show their abilities because they can 
distinguish between methods of doing a real and virtual practicum. The results of this study are the validity of authentic 


https://doi.org/10.33225/jbse/23.22.631 QT 


637 


Journal of Baltic Science Education, Vol. 22, No. 4, 2023 


THE FEASIBILITY AUTHENTIG ASSESSMENT INSTRUMENT THROUGH VIRTUAL LABORATORY ISSN 1648-3898 | /print/ 
LEARNING AND ITS EFFECT ON INGREASING STUDENTS’ SCIENTIFIG PERFORMANGE 
(PP.63 1-640) ISSN 2538-7138 /Online/ 


assessment instruments that can increase students’ critical thinking skills by 0.532 points, with a significance and R2 
of 0.00 and 0.974, respectively (Damayanti et al., 2017). In addition, higher-order thinking and real-world problem- 
solving are two important elements in the development of authentic assessment (Azim, 2012; Basori & Mubarok, 2020). 
Several other reports also show that performance appraisal improves students’ competency in practicum because 
rubric indicators represent all aspects of practicum implementation. These indicators are then able to overcome the 
problems encountered in biology practicum activities (Ramadhan & Suyanto, 2020; Sudrajat, 2012). In the virtual 
laboratory learning process, the increased scientific achievement is shown by students directly completing tasks 
through the applications provided at the stage of formulating experimental variables. Completion of assignments is 
also emphasized in investigations carried out by students starting from the pattern of determining the pH. Through 
the application of PhET simulations on https://phet.colorado.edu/ in a virtual laboratory, it is possible to distinguish 
strong acids from weak acids as shown through project appraisal analysis using a portfolio. There are obstacles in 
carrying out authentic assessments using virtual laboratories, where only some teachers understand the technology 
and run virtual laboratory applications, so it takes a long time when learning takes place. Even so, the final grades of 
students increased even though the school was experiencing limitations in providing practicum tools and materi- 
als. This is because students are enthusiastic about getting this new learning model. In line with Ali et al. (2022), an 
alternative practicum is carried out with using materials and limited laboratory equipment. 

The use of authentic assessment through virtual laboratory learning is expected to be able to provide active 
learning solutions to improve student learning outcomes. Authentic assessment through the virtual laboratory is 
relevant enough to be implemented so that it becomes a solution to limited learning resources. The process of virtual 
laboratory activities is also a part that needs more attention, considering that the interface in the virtual laboratory 
is the only form of communication between the practitioner and the teacher. The existence of a proper and system- 
atic assessment process will greatly influence the conclusion, namely whether the learning objectives are achieved. 


Conclusions and Recommendations 


The findings of this study reveal that the application of authentic assessment through virtual laboratory learning 
is appropriate for use by teachers in schools, especially in expressing students’ scientific performance. The scientific 
performance of students has increased significantly with the high category on several indicators displayed. These 
results make a positive contribution to revealing students’ skill abilities during virtual practicum activities. This find- 
ing also has a real impact on teachers who implement virtual laboratories in class. Virtual laboratories can display 
real theories and experiments without direct application in the laboratory room. The advantage is that teachers can 
save time in giving material to students. Even though there are several obstacles, namely only some teachers who 
understand technology, this makes a separate motivation for teachers in developing learning innovations. The use 
of complete and flexible assessments can make it easier for teachers to conclude students’ final grades. However, in 
the process of applying the assessment, the teacher takes a long time to fill in the value manually. Finally, it can be 
recommended to use technology in implementing assessments, especially performance assessments so that they 
focus more on improving students’ scientific performance and shortening the teacher's time in the assessment process. 


Limitations 

This research is limited to the feasibility test of the authentic assessment that is used, namely only observa- 
tion sheets of assignments and portfolios as well as scores for improving the scientific performance of students 
studying in 11th-grade. The virtual laboratory used is also limited to acid-base material using the PhET simulation 
application on https://phet.colorado.edu/. 
Declaration of Interest 
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Introduction 


The era marked by the development of worldwide trends is recognized 
as Society 5.0. Studies by Rodriguez-Abitia & Bribiesca-Correa (2021) and 
Suryantini et al. (2020) have established a connection and correlation 
between real and virtual space and Society 5.0. The cycle of data acquisi- 
tion, evaluation, and translation into pertinent knowledge, subsequently 
applied in the real world, has been identified as a recurring feature of 
Society 5.0. This cycle has been proposed to operate on a societal level 
as well (Deguchi et al., 2020). The transformation of society will invariably 
lead to changes in the educational landscape. The primary concerns are the 
skills that students will gain and the courses that will be offered. Research 
has highlighted creativity (Carayannis & Morawska-Jancelewicz, 2022), 
soft skills (SA & Serpa, 2022), higher-order thinking skills (Suryantini et al., 
2020), communication (Joko et al., 2023), collaboration (Sa & Serpa, 2022), 
information and media literacy (Sa et al., 2021), computing and ICT literacy 
(Smuts & Van der Merwe, 2022), all of which mirror 21st-century skills, as 
leading elements to be imparted to students. 

The acquisition of these skills has been the topic of educational study 
for decades (Chalkiadaki, 2018; Turiman et al., 2012; van Laar et al., 2017). 
According to several studies (Fakaruddin et al., 2023; Miwa, 2020; O’Neal 
et al., 2017), the learning of these abilities should begin in early grades and 
be integrated throughout numerous courses. The contribution of science 
education to the acquisition of these skills is undeniable. In the study con- 
ducted by Berg et al (2021), out-of-school science education suggestions 
have been developed to help students cultivate 21st-century skills. In order 
to develop 21st-century skills, many projects are carried out in the science 
courses (Chu et al., 2017). 

One of the fundamental courses in elementary school that serves to 
build a strong foundation for essential skills is the Science and Technology 
course (STC). In elementary schools, the STC curriculum introduces stu- 
dents to a broad range of general science topics, carefully tailored to suit 
their cognitive development levels. This approach ensures that students 
are taught with appropriate content that enables them to understand 
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Abstract: Science education in primary 
school is critical for establishing a 
comprehensive understanding of the 
nature of science. Students with positive 
attitudes towards science courses are 
more likely to achieve academic success. 
Measuring and fostering positive attitudes 
is essential for effective instructional 
planning. This study aimed to develop a 
measurable scale to assess the attitudes 

of fourth-grade elementary school 
students in Northern Cyprus towards 
Science and Technology Courses (STC). 
While developing the Student Science 

and Technology Attitude Scale (SSTAS), 

the literature was reviewed, and 168 
students were asked to write essays about 
STC. A pool of attitude items was created 
by analyzing the literature and essays, 

and expert opinions were sought. The 
preliminary scale was prepared based 

on expert consultation. The study group 
included 651 randomly selected fourth- 
grade students during the 2020-2021 
academic year. Exploratory Factor Analysis 
(EFA) and Confirmatory Factor Analysis 
(CFA) were conducted to determine SSTAS's 
construct validity. The results indicated that 
the scale is valid and reliable, comprising 
three dimensions. The first dimension 
measures students’ attitudes towards the 
teaching process, with 13 items. The second 
dimension assesses students’ negative 
attitudes, including five items. The third 
dimension evaluates students’ attitudes 
towards experiments, containing five items. 
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the world. Fitzgerald and Smith (2016) emphasize the critical importance of maintaining consistent teaching 
and learning standards throughout the science curriculum. This consistency is necessary for effective scien- 
tific knowledge in the broader context of student's intellectual development, enrichment of their reasoning 
abilities, comprehension skills, and overall cognitive talents. By doing so, educators can create a more holistic 
educational experience that fosters the development of well-rounded individuals. Science education plays a 
vital role in cultivating environmentally conscious, inventive, curious, and imaginative citizens within the society 
(Hayrynen et al., 2021; Schonfelder & Bogner, 2020). With the help of science and technology concepts, students 
acquire a deeper appreciation for the natural world, learn to think critically about complex issues and develop 
problem-solving skills that can be applied to various aspects of their lives. This education empowers students 
not only to understand scientific principles but also to engage actively with their environment and contribute 
positively to the world around them. 

The most important aim of STC is to inform students about current scientific developments and to raise 
generations that can benefit from all fields and at all levels of science. Studies related to science education and 
teaching are of great importance for the future of humanity. 

A lesson on contemporary science teaching should address the needs and interests of the student (Ornek 
& Saleh, 2012). It may be tailored in accordance with the child’s stage of development and is taught using ap- 
propriate approaches that take environmental variables into account (Hansson et al., 2021). In contemporary 
pedagogical approaches within the sphere of scientific education, the emphasis has increasingly shifted towards 
the imperative of cultivating students’ capacity for scientific reasoning and critical inquiry, rather than a sole 
focus on the comprehensive impartation of established scientific knowledge (Fakaruddin et al., 2023). The ob- 
jective of this approach is to facilitate students’ acquisition of skills vital to scientific inquiry, including research 
conduction, data analysis, and synthesis. The knowledge acquired through Science, Technology, and Society 
(STS) curriculum not only enhances these academic competencies but also equips students with practical skills 
applicable in their everyday lives. This multifaceted educational experience fosters an overall academic readi- 
ness, thereby simplifying the learning process for additional subjects. Therefore, science education serves a dual 
purpose: it not only imparts scientific knowledge but also cultivates an overarching learning skill set in students 
(Letina, 2020; Mateos-Nufez et al., 2020; Pontinen & Raty-Zaborszky, 2020). 

Science education in elementary school serves as a crucial foundation for students, fostering a compre- 
hensive and consistent understanding of the nature of science. This early exposure to scientific concepts and 
principles is crucial for preparing students to become scientifically literate, a key outcome of science education 
(Roberts, 2007). Scientific literacy equips students with the knowledge, skills, and mindset necessary to engage 
with and interpret scientific information in their everyday lives. In addition, elementary school science educa- 
tion is essential for igniting students’ curiosity about the scientific world (Fitzgerald & Smith, 2016; Jufrida et al., 
2019). By introducing engaging and age-appropriate content, educators can inspire students to ask questions, 
investigate concepts, and develop a genuine interest in scientific inquiry. This enthusiasm for learning can then 
be carried over into later years, encouraging students to pursue further education in science-related fields and 
contribute to the advancement of scientific knowledge. A comprehensive science education in elementary 
school helps students develop skills in critical thinking, problem-solving, and analysis. Students are equipped to 
navigate complex issues and make informed decisions in a technologically advanced and interconnected world 
by acquiring these competencies. According to Harlen and Qualter (2004), elementary school science teaching 
has a helpful role in the formation of attitudes along with the development of students’ ideas and skills. In this 
respect, elementary school teachers need to create teaching processes in which students will participate at a high 
level. The capacity of elementary school instructors to design scientific courses that are engaging for children 
and that pique students’ interest about the outside world has a significant effect in kids’ improved involvement 
in science classes (Harrison, 2020). For this reason, it is essential for elementary school teachers to prepare fun 
teaching processes that will arouse students’ interest in STC topics, attract their attention, and enable students 
to be active. With such environments, it will be easier for students to learn, and it will also serve to create posi- 
tive attitudes towards science starting from an early age. 

The achievement of goals and behaviors in the Science and Technology course (STC) is strongly linked to the 
course success and the students’ attitudes towards it. Attitude has been defined by numerous scholars in various 
ways. Thurstone (1931) described the attitude as a state of readiness to exhibit a positive or negative response 
towards objects and symbols (Tavsancil, 2010). In contrast, Katz (1967) characterized attitude as a preliminary 
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thought process wherein an individual perceives a symbol, object, person, or the world as either good or bad, 
beneficial or harmful, based on their personal value system. 

Attitude can be characterized as an acquired phenomenon that significantly impacts an individual's be- 
havioral patterns, potentially introducing a degree of subjectivity or bias into their decision-making processes 
(Eaton et al., 2008). Eagly and Chaiken (2007) defined attitude as a psychological tendency expressed through 
the evaluation of a specific entity in either a favorable or unfavorable manner. There is a broad agreement among 
researchers that attitude represents a summary assessment of a psychological object along dimensions such as 
good-bad, harmful-beneficial, pleasant-unpleasant, and likable-unlikable (Ajzen, 2001). 

When learning new information, an individual needs to possess some prior knowledge about the subject 
matter. However, the interest and curiosity that the individual displays toward learning the material are equally 
significant (Hidi & Renninger, 2020). Consequently, fostering positive attitudes towards subjects like the STC is 
essential to enhance students’ engagement, curiosity, and overall success in the learning process. 

Affective entry behaviors consist of dimensions related to feelings such as interest, motivation, anxiety, 
and academic self-design. Children attitudes begin to take shape at a young age (Halbeisen et al., 2017), making 
them a crucial factor in the development of good behavior toward STC at the elementary school level (Harlen 
& Qualter, 2004; Kerr & Murphy, 2012). 

Recent research has demonstrated that students with good attitudes about science courses are more likely 
to achieve academic achievement than those with negative attitudes. For instance, Osborne et al. (2003) discov- 
ered that students with good attitudes toward science were more likely to get higher grades in scientific classes 
than those with negative views. There are several reasons why attitudes toward scientific courses may influence 
academic performance. Students with positive attitudes about scientific courses are more likely to participate 
in learning activities and be motivated to study, resulting in improved academic success (Linnenbrink-Garcia et 
al., 2016; Teppo et al., 2021). Good attitudes toward science may enhance self-efficacy, or the idea that one can 
succeed in science courses, which can result in improved effort and perseverance (Gonzalez-Gomez et al., 2022; 
Mao et al., 2021; Retni S et al., 2021; Ultay et al., 2020). Positive attitudes about scientific courses may result in 
a stronger interest in science-related occupations, which can drive students to do better in their classes (Mao 
et al., 2021). As conclusion, there is a considerable correlation between attitudes toward scientific courses and 
academic performance. The use of engaging and dynamic teaching techniques, the provision of chances for 
hands-on learning, and the promotion of science’s relevance to students’ everyday lives may all contribute to 
the development of favorable attitudes toward scientific courses. 

Due to societal prejudices, many students perceive STC to be confusing and difficult to comprehend. This 
misconception impacts students’ perceptions of science and technology and, consequently, their academic 
performance (Ogunkola & Samuel, 2011; Patall et al., 2018). Consequently, scholarly recommendations advocate 
for the early exposure of children to scientific principles and suggest fostering an enjoyment for the subject. 
This approach is proposed as an effective strategy for the development of positive and constructive attitudes 
towards science (Stagg & Verde, 2019; Teppo et al., 2021). By developing a positive attitude towards science, it 
is possible to increase students’ interest in this field and to direct them to professions related to science in the 
future (George, 2006). In this respect, attitudes are one of the most effective elements in learning. 

According to Bentley et al. (2007), the attitude dimension of science is the dimension of feeling and value, 
and it is the driving force of learning. In addition, the interest and curiosity of the student generate research 
and then provide the construction of new knowledge (Bentley et al., 2007). 

In the Northern Cyprus elementary school program, STC subjects are included within the Life Sciences 
courses in the 1st, 2nd, and 3rd grades, while they are included as Science and Technology Courses in the 4th and 
5th grades. In the 2003-2004 academic year, within the scope of the Basic Education Support project, a decision 
was made to change the name of the Primary Education Science course to STC, and it was put into practice in 
the 2004-2005 academic year (MEB, 2004). The Northern Cyprus Education System was revised in 2005 under the 
name of the New Education System on the basis of constructivism (KKTC MEB, 2005). Elementary education cur- 
ricula were reconsidered in 2016 and rewritten based on the constructivist approach (http://tepgp.emu.edu.tr/). 

In this context, the objective is to create a valid and reliable measurement instrument specifically designed 
to assess the attitudes of elementary school students toward the Science and Technology course. Developing 
such a tool would provide valuable insights into students’ perceptions, feelings, and preferences regarding the 
course, enabling educators and researchers to better understand and address factors that may influence their 
engagement, motivation, and overall academic performance. 
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Research Design 


In this study, the scale development process was carried out and this process involved a quantitative research 
approach to construct a comprehensive and robust instrument. The primary aim was to create a scale that effectively 
captures various aspects of students’ attitudes toward Science and Technology Courses (STC), allowing a deeper 
understanding of their perceptions and feelings. In addition to scale development, the study also aimed to assess 
and establish the psychometric properties of the instrument, ensuring validity and reliability. By concentrating on 
both the scale generation process and the examination of its psychometric features, the study aimed to give educa- 
tors, academics, and policymakers a helpful and trustworthy instrument for measuring students’ views regarding 
STC. In return, this would guide instructional planning, curriculum development, and interventions to encourage 
positive attitudes and improve students’ academic performance in scientific education. 


Study Group 


The research study's participant group comprised 651 fourth-grade students attending private elementary 
schools under the North Cyprus Ministry of National Education and Culture during the 2020-2021 academic year. 
Information was not collected solely on the sociodemographic description of the participants. While opinions may 
vary regarding the ideal sample size for scale development studies, it is generally recommended that the sample 
size be five to ten times larger than the number of items in the scale (Child, 2006; Cokluk et al., 2014; Williams et al., 
2010). Since there are 35 questions in the first form of the scale, the sample size must be at least 350. In this case, 
it can be said that the sample size is sufficient since 651 people participated in the scale development process. 


Scale Development Process and Data Collection 


During the development of the scale, the literature (Dijkstra & Goedhart, 2012; Summers & Abd-El-Khalick, 
2018; Yurdakal & Kirmizi, 2019) was scanned to create an item pool and examined. At the same time, the answers of 
168 students to open-ended questions via Google forms were analyzed. By analyzing the answers of each student, 
the researchers determined the frequencies of expressions that could be potential attitude expressions. Based on 
the literature review and analyses, an item pool consisting of 40 items was created. Every effort has been made to 
ensure that the items are clear and comprehensible, and that each item conveys a unique opinion. 

The item pool was examined by a Curriculum and Instruction, linguist, psychological counseling, and guid- 
ance specialist, as well as elementary school teachers. An overview was taken in terms of suitability for the levels 
of student development, language, curriculum development, and assessment and evaluation. Two of the draft 
items were found to be problematic in terms of language, and three of them were removed from the scale as 
they were found to be problematic in terms of suitability for the student's level, and an initial form consisting of 
35 items was prepared. 

To assess the level of agreement among respondents for the items in the scale, a 5-point Likert-type scale 
was utilized, with response options ranging from “Strongly agree” to “Strongly disagree,’ with “Agree,"“Undecided,” 
and “Disagree” in between. The scale comprises 35 items, of which 19 are positive and 16 are negative. Students 
responded to the items using a 5-category rating scale. For positive attitude items, scoring was assigned as follows: 
“Strongly agree” = 5, “Agree” = 4, “Undecided” = 3, “Disagree” = 2, and “Strongly disagree” = 1. For negative items, 
the scoring was reversed: “Strongly agree” = 1, “Agree” = 2,“Undecided” = 3, “Disagree” = 4, and “Strongly disagree” 
= 5. The attitude scale was converted into a Google Form and distributed to fourth-grade students. A total of 697 
students participated in the study by responding to the survey. This method allowed for the efficient collection 
and analysis of data, helping to understand better the students’ attitudes toward the subject matter in question. 


Data Analysis 
Analysis of the data was carried out to provide evidence for the construct validity of SSTAS, “Exploratory Factor 


Analysis (EFA)” and“Confirmatory Factor Analysis (CFA)”. SPSS 21.0 package program was used for EFA.“Kaiser Meyer 
Olkin Test (KMO)” and “Barlett Sphericity” test methods were used to determine whether the data were suitable for 
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principal component analysis. Varimax transformation was used to determine the factors so that the factors to be 
obtained are not correlated with each other. The factors that emerged as a result of the analysis were named. For 
CFA, the Lisrel 8.8 package program was used, and the suitability of the model revealed in EFA was checked. For 
the content validity of the scale, the opinions of academicians who are experts in their fields were consulted. To 
determine the reliability of SSTAS, the Cronbach-Alpha test and composite reliability are calculated. 


Research Results 


Exploratory factor analysis and confirmatory factor analysis were applied to examine the construct validity 
of the SSTAS, and the findings of the applied factor analyses are as shown below; 


Exploratory Factor Analysis 


Before initiating the Exploratory Factor Analysis (EFA) procedures, the dataset was thoroughly examined 
to ensure that it met the necessary assumptions for conducting factor analysis. In this regard, the researchers 
first assessed the draft form of the Student Science and Technology Attitude Scale (SSTAS) for its compatibility 
with the multivariate normal distribution. Upon examination, it was confirmed that the draft form conformed to 
the multivariate normal distribution requirements. Next, the Kaiser-Meyer-Olkin (KMO) coefficient and Bartlett’s 
sphericity tests were employed to ascertain the dataset's suitability for performing EFA. The SSTAS yielded a KMO 
coefficient of .934, which provides valuable insights into the data matrix’s appropriateness for factor analysis and 
the data structure's suitability for factor extraction. For a dataset to be considered factorable, the KMO coefficient 
should be higher than .60. According to Cokluk et al. (2014), KMO values of .90 or greater are deemed ‘perfect’ Ad- 
ditionally, the Bartlett test was used to examine the presence of relationships between variables based on partial 
correlations (Cokluk et al., 2014; Williams et al., 2010). Upon analyzing the results of Bartlett's Test of Sphericity, the 
researchers found that the test’s calculated chi-square value was 5333.200, which was statistically significant (p < 
.05). Based on these findings, it was concluded that the SSTAS was appropriate for EFA, allowing the researchers 
to proceed with confidence in their analysis. 


Figure 1 
Scree plot of SSTAS 
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Figure 1 shows the scree plot used to have an idea about the factor structure of SSTAS, and it was concluded 
that the scale has a 3-factor structure. 
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SSTAS Exploratory Factor Analysis Results 


F1 F2 F3 
Eigenvalues 7.571 2.566 1.418 
Explained variance (%) 32.918 11.157 6.164 
Cumulative explained variance (%) 32.918 44.075 50.239 
20. Science class is good. 817 
15. | enjoy doing science homework. 804 
35. | would like to take a science class. 137 
31. | would be happy to increase the course hours of the Science course. 71 
3. Science class is fun. .690 
1. | love science class. .676 
24. Science lesson is easy. 671 
33. | think everyone should like Science class. 665 
17. Science class is boring. 664 
21. | find science class boring. 648 
29. Science lesson is practical. 623 
32. Studying for science class makes me uncomfortable. 614 
18. Science lesson teaches us the information we want to learn. 514 
6. The knowledge we learn comes in handy in everyday life. 500 
5. The subjects of the science lesson are difficult. 497 
10. | don’t like studying science. 157 
14. | do not like learning by doing science lessons. 126 
19. | don't like science and technology in science class. 710 
28. | don't like reading books about science class 669 
16. We do not learn different information in science class 551 
25. Experiments are interesting. 125 
4. Experiments do not make science class fun. 634 
12. | will not be happy if the experiments multiply. 606 
27. It makes the laboratory science lesson more beautiful. 606 
13. Experiments ensure the permanence of knowledge. 0.516 516 


Table 1 presents the eigenvalues obtained from the Exploratory Factor Analysis (EFA) applied to the Student 
Science and Technology Attitude Scale (SSTAS), along with the variances explained by these eigenvalues both 
individually and cumulatively. In the EFA, the Principal Components Analysis method was employed to identify 
the factor structure of SSTAS, and a varimax transformation was conducted on the dataset to enhance the inter- 
pretability of the results. A close examination of the factor loadings of the items in STAS revealed that they ranged 
from .31 to .81. As per Tabachnick and Fidell (2007), a minimum factor loading of .30 is expected for an item ona 
factor. As a result, 10 out of the 35 items in the SSTAS were eliminated from the scale due to factor loadings below 
the acceptable threshold. 

For the 25-item version of STAS, the factor loadings of the items varied from .497 to .817. Based on these 
findings, it was determined that the SSTAS consisted of a 3-factor structure, which provided a comprehensive 
and coherent framework for assessing elementary school students’ attitudes toward the Science and Technology 
Course. The factor names were as follows: 
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Factor 1: Teaching Process (Items: 1, 3, 5, 6, 15, 17, 18, 20, 21, 24, 29, 31, 32, 33, 35) 

Factor 2: Negative Attitudes (Items: 10, 14, 16, 19, 28) 

Factor 3: Experiments (Items: 4, 12, 13, 25, 27) 

This 3-factor structure provided a clear and organized framework for understanding and interpreting the 
elementary school students’ attitudes toward the Science and Technology Course. 


Confirmatory Factor Analysis 


Upon identifying the factor structure of the Student Science and Technology Attitude Scale (SSTAS), Con- 
firmatory Factor Analysis (CFA) was implemented to validate the appropriateness of the factor structure and to 
scrutinize the relationships between the factors. CFA is considered an extension of EFA, as it serves to verify the 
factor structure established during the exploratory stage. CFA is a valuable tool in the scale development process, 
as it allows researchers to test the hypothesized factor structure based on the results of the EFA against the actual 
data. By doing so, it provides empirical evidence for the construct validity of the scale and ensures that the factor 
structure is robust and generalizable to the target population. Furthermore, CFA helps in determining the extent to 
which the factors are correlated, enabling researchers to gain a better understanding of the relationships between 
the factors and their underlying constructs. In the context of the SSTAS, applying CFA was essential to confirm that 
the three-factor structure, which included the Teaching Process, Negative Attitudes, and Experiments factors, ac- 
curately represented the attitudes of elementary school students towards the Science and Technology Course. By 
confirming the suitability of the factor structure, researchers could ensure that the SSTAS was a valid and reliable 
tool for measuring students’ attitudes, allowing for more effective assessment, intervention, and improvement of 
their learning experiences in the Science and Technology Course. 


Figure 2 
Confirmatory Factor Analysis Path Diagram of SSTAS 
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The CFA path diagram of the SSTAS shown in Figure 2 is given, and it was discarded from the 3-factor and 25- 
item scale determined by EFA because the 2-item factor loads were not appropriate and disrupted the model fit. 
Accordingly, in the final form of the scale, a total of 23 items were included in the Science and Technology Lesson 
Teaching Process sub-dimension 13 items, Negative Attitudes Science and Technology Lesson sub-dimension 5 
items, and Experiments in the Science and Technology Lesson sub-dimension 5 items. 


Table 2 
SSTAS Goodness-of-Fit Indices for SSTAS 


Factor/Dimension y7/SD GFI NFI CFI RMSEA 
Criteria < 3 (perfect fit) > .90 > .90 > 90 .05 and .08 (acceptable fit) 
Model 2.693 916 .904 .923 53 


When the SSTAS, CFA goodness-of-fit index values (as shown in Table 2) were examined, it was determined 
that y’/°° was 2.693, which indicates that SSTAS has a perfect fit in terms of y7/°°. According to Kline (2005), a value 
of y’/sd below 3 indicates a perfect fit, and a value between 3 and 5 indicates an acceptable fit. 

According to SSTAS and DFA results, the Goodness of Fit Index (GFl) value of the scale was .916, the Normal- 
ized Fit Index (NFI) value was .904 and the Comparative Fit Index (CFI) value was .923. According to Tabachnick 
and Fidell (2007), the limit value determined for GFI, NFl and CFI is between .90 and 1.00. GFI, NFIl and CFI values 
are expected to be between the specified limit values, and this indicates an acceptable fit. Root Mean Square Error 
(RMSEA) value of the scale was found to be 0.053. According to Brown (Brown, 2015), a RMSEA value between .00- 
.05 indicates perfect fit, and a value between .05-.08 indicates acceptable fit. Accordingly, it has been determined 
that SSTAS has an acceptable fit in terms of RMSEA. 

As a result of the CFA, it was determined that all goodness-of-fit indices of SSTAS except GFI were appropriate 
and the construct validity of the scale was ensured. 


SSTAS Reliability Analysis 


In order to examine the reliability of the SSTAS, the Cronbach's alpha test and the split-half test were performed, 
respectively, and the item-total correlations of the scale were examined. 


Table 3 
Reliability Results for Sub-dimension and Total Scale 


Cronbach’s a McDonald’s w 
Teaching Process 900 .904 
Negative Attitudes .154 157 
Experiments 693 697 
Total 877 898 


Upon examining Table 3, it was determined that the Cronbach's alpha coefficient for the overall Student 
Science and Technology Attitude Scale (SSTAS) was .877. Additionally, composite reliability (McDonald's w) was 
calculated for each dimension and the total scale to further assess the internal consistency of the scale. According 
to Buyukoztirk (2015), a Cronbach's alpha reliability coefficient above .70 is considered an acceptable level, while 
Taber (Taber, 2018) suggests that a coefficient above .60 is adequate for scale reliability. 

Considering these criteria, the reliability coefficients of the SSTAS's sub-dimensions and the total scale are 
within acceptable levels, indicating that the scale exhibits satisfactory internal consistency. This internal consistency 
is crucial in the context of academic research, as it provides evidence that the items within each dimension of the 
SSTAS are measuring the same underlying construct, leading to more accurate and reliable results. 
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Having a reliable scale is vital for assessing students’ attitudes toward the Science and Technology Course, as it 
ensures that the scale can consistently and accurately capture the attitudes of the target population. Consequently, the 
SSTAS can be considered a valuable tool for educators, researchers, and policymakers in evaluating the effectiveness of 
instructional practices and in informing potential improvements in science education for elementary school students. 


Discussion 


This research was conducted to develop a valid and reliable attitude scale to determine the positive and nega- 
tive attitudes of elementary school students toward STC. Attitude scales are measurement tools used to determine 
students’ attitudes toward the courses. An attitude scale has been reached to determine the attitudes of secondary 
school students towards STC in the TRNC. However, a scale aiming to measure elementary school students’ attitudes 
toward STC could not be found in the North Cyprus. In the literature, there exists a scale for School Improvement to 
Transform Education into being sustainable environments but not about STC in North Cyprus (Mafratoglu et.al, 2023). 
Therefore, this study can be accepted as a contribution to the literature in determining the positive and negative 
attitudes of elementary school students toward STC. The validity of each item was verified by the fact that a broad 
variety of publications were researched in the process of generating the item pool, and that the items themselves 
were evaluated by specialists in the relevant fields. Expert evaluation is one of the first steps to be implemented in 
scale development (Koc & Budak, 2021; Zapata-Ospina & Garcia-Valencia, 2022). The scale was prepared in a 5-point 
Likert type. During the scoring of the items in the scale, negative items should be scored in reverse. 

Exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) were used to determine the construct 
validity of the developed scale. It has been observed that the findings obtained serve to measure the feature which 
the scale aims to measure and can distinguish individuals who have the desired feature to be measured and those 
who do not. EFA is among the multivariate statistical techniques used to determine the latent sub-dimensions of the 
scale (Watkins, 2018). The scale consists of 23 items and three factors. Three factors explain 52.23% of the total vari- 
ance of the attitude variable. The first factor constitutes 32.9% of the scale, the second factor 11.16% and the third 
factor 6.16%. The total explained variance is over 50%. 

The results of exploratory factor analysis (EFA) indicated reasonable (.50 to.90) and significant (/p < .001) coefficients, 
thus a confirmatory factor analysis was carried out in order to validate the three-component structure that EFA had 
discovered. This indicates that each item makes a substantial contribution to the subscale that is being considered. 
The number that was derived from the ratio of proposed chi-square to degree of freedom was employed in order to 
establish whether or not the model was a good fit. The results of this investigation are presented in Table 2, where it 
can be seen that the ratio of chi-square to degrees of freedom is 2.693. According to Kline (2005), a perfect match is 
indicated by a value of y’/sd that is less than 3, whereas an adequate fit is indicated by a value that falls between 3 
and 5. In light of these standards, it is possible to assert that the model provides a satisfactory fit for the data. It was 
discovered that the NFI for the model was equal to .91, and the CFI was equal to .93. According to Brown (Brown, 
2015), a perfect match is indicated by an RMSEA value of between .00 and.05, while a good fit is indicated by a value 
between .05 and .08. As a result, we may deduce that the value of the RMSEA for the Science and Technology Lesson 
Attitude Scale is .053, indicating that it has an adequate fit. 

When the results of the reliability analysis are examined, it is concluded that it is reliable at an acceptable level 
in terms of both general and sub-factors. The Cronbach Alpha internal consistency coefficient of the whole scale was 
found to be .87. In structural equation modeling (SEM), the concept of composite reliability refers to a statistic that 
evaluates the internal consistency of a latent variable or construct (Brunner & SUR, 2005). McDonald's w is calculated 
for composite reliability and coefficient (.898) is at acceptable level. 

Itis highlighted that there are various structures for assessing the attitude toward the science course, as opposed 
toa single structure (George, 2006; Kerr & Murphy, 2012; Osborne et al., 2003). Hence, scales may contain several fac- 
tors. This concept is supported by the emergence of a triple-factor structure in this investigation. According to Williams 
et al. (2010), the process of labeling factors is a complex and nuanced task that requires researchers to balance their 
subjective interpretations, theoretical knowledge, and inductive reasoning to assign meaningful and coherent labels 
to the factors identified in their analysis. This process is crucial for ensuring that the results of factor analysis are both 
meaningful and relevant to the broader context of the research question being investigated. 

Although other names are feasible for the elements in factor 1, it is believed that “teaching process” is more in- 
clusive because it symbolizes the teaching process. In a research study involving the development of an attitude scale 
toward science, attitude-related questions were frequently titled “Attitude toward science” (Summers & Abd-El-Khalick, 
2018). In as much as the research in question contains elements connected to belief, naming demonstrates differ- 
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ence along other dimensions. When the elements of Factor 2 were analyzed, they were termed “Negative Attitudes 
since they were entirely negative. In a different research study (Liaghatdar et al., 2011), negative items were labeled 
“Science Anxiety.’ The third aspect in our analysis was titled “Experiments.’In another research (Nuhoglu, 2008), it was 
referred to as “class activities.’ ln essence, although the naming of the scale items is considered subjective, there are 
similar namings in the literature, and the fact that they were called to correspond with the items gave the meaning 
of the scale’s components. 


Conclusions and Implications 


This scale was designed to assess the attitudes of elementary school students about the science and technology 
curriculum. Based on the outcomes of the EFA, CFA, and reliability analyses, the scale can be argued as valid and reli- 
able. In its final form, the scale has three dimensions. The first dimension evaluates the attitudes of students toward 
the teaching process. This dimension has 13 components, of which 3 are negative and 9 are positive. The second 
dimension evaluates the negative views of the students. This dimension has a total of five negative items. The third 
dimension evaluates the students’ perspectives on the experiments. There are a total of five items in this dimension, 
one of which is negative. 

Attention should be paid to the fact that all items were marked by the students and that the negative items were 
coded in reverse during the assessment stage when using the scale. Using different populations, future researchers 
can evaluate the validity and reliability of the scale. It is also adaptable to other cultures and languages. The main 
limitation of the study is that EFA and CFA were performed on the same data set. Due to the use of the same dataset as 
researchers, each step was checked for overfitting, capitalization on chance and inflated fit indices. In particular, it was 
checked whether there were idiosyncrasies in the data. The reliability of the scale can be more clearly demonstrated 
by future researchers’ studies on different data sets. 
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Introduction 


Energy is a key concept in science education and in our everyday life. 
At the same time, current everyday energy production is also affecting our 
everyday life and one of the biggest challenges facing society today is the 
need for a paradigmatic shift in climate policy (Hall, 2007; Kinley, 2017; Meck- 
ling & Allan, 2020; Nepras et al., 2023; United Nations, 2015). According to the 
European Green Deal (European Commission, 2019), the goal is to enable no 
net greenhouse gas (mainly CO,) emissions throughout the European Union 
by 2050, in order to stop further global warming (European Commission, 
2019). In recent years, carbon footprint is considered as a main metric indica- 
tor in greenhouse gas emissions (Kanemoto et al., 2016; Schwenkenbecher, 
2014; Yue, 2020). Main carbon footprint increasers are fossil fuels, which are 
currently used in energy production and transportation (Shea et al., 2016). In 
order to reduce governmental carbon footprint, Estonia's biggest challenge 
is definitely replacing oil shale electricity production with alternative energy 
sources (Kama, 2016; Holmgren et al., 2019). In this respect, the Estonian Edu- 
cation Strategy for 2021-2035 (EMER, 2020) indicates that steps are needed 
to be taken to achieve climate neutrality, by raising awareness related to 
the state of the environment, sustainable development, and enhancing an 
energy-efficient infrastructure. 

Based on the above, Ocetkiewicz et al. (2017) have raised a concern 
whether energy conceptualizing and sustainability are sufficiently promoted 
in education. Multiple authors have indicated that educators need to raise 
students’awareness of future sustainability concerns on planet Earth (Bilgen 
& Sarikaya, 2018; Klein & Coffey, 2016; Murray, 2012; Rieckmann, 2018; Sidi- 
ropoulos, 2014; van Ruijven et al., 2019). Given the objectives of the energy 
sector for the next thirty years (Skea et al., 2011; European Commission, 2019), 
a key challenge in the field of education is to develop educated workers 
conceptualizing energy and especially energy transfer and energy transfor- 
mation in line with labor market expectations (Kandpal & Broman, 2014). 
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Abstract. This study on the Estonian lower 
secondary (7-9th grade) science curricula, 
is seen as an important framework 

for educators preparing students as 
tomorrow’ citizens able to reflect on 
sustainable energy development. As 

the curriculum is taken to be the major 
document allowing insights into Estonian 
educational standards, this study identifies 
components within the intended lower 
secondary science curricula for subjects 

of biology, chemistry, earth science, 
physics, and interdisciplinary science. 
Using document analysis, verbs associated 
with career-related learning outcomes 

are detected, allowing the relatedness 

of the energy conceptualizations and 
determination of their cognitive level 
utilizing SOLO (Structure of Observed 
Learning Outcomes) taxonomy. A team 

of coders identify a total of 782 learning 
outcomes across three learning domains: 
psychomotor (176), affective (32), cognitive 
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(225), relational (276), and extended 
abstract (40) levels. The majority of energy 
concept learning outcomes (274) are 
identified in the source (form) and transfer 
(transform) categories. Very few career- 
related learning outcomes are detected 
with the science education relevance 
dimensions (individual, societal, career). 
The suitability of the findings is discussed. 
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applied to other educational disciplines 
for raising awareness of disciplinary 
crosscutting concepts. 
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The main framework used by educational institutions for orienting students’ learning is the curriculum (McFar- 
lane & Ogazon, 2011). The intended curriculum parameters — referred to as learning outcomes are usually written 
related to three different domains: cognitive, affective, and psychomotor (S6nmez, 2017). And within the curriculum, 
the science education provision (in the various science sub-divisions) focuses on energy transfer (Eisenkraft et al., 
2014; Fortus et al., 2015; Guven & Cakir, 2019; Next Generation Science Standards, 2012; Park, 2019). Suryana et 
al. (2019) indicate that students tend to lack cognitive knowledge of the energy learning process. Nevertheless, 
studies show that teaching related to the concept of energy is problematic (Ben-Zvi e al., 1993; Driver et al., 2014; 
Herrmann-Abell & DeBoer, 2018; Mamlok-Naaman & Mandler, 2020), both within and across science education 
subjects, even though issues associated with energy need to be meaningfully developed for preparing future 
well-educated citizens (Hodson, 2011; Matus & Talburt, 2015; Yacoubian, 2015). 

Relevance is one of the key terms in science education and making science education teaching and learning 
processes and future science-related career relevant needs to be a key goal (Stuckey et al., 2013; Kim & Dionne, 
2014; Lamanauskas, 2022) in the development of scientific literacy, seen as going beyond the acquisition of cogni- 
tive learning and relating to learning at both societal (linking science to the concerns within the society) and the 
acquisition of personal attributes such as problem solving and decision making (Holbrook & Rannikmde, 2009). 
Thus, in order to determine whether the current provision for energy studies is appropriate in preparing students 
for future changes in energy developments, there is a need to determine whether (the intended curriculum) meets 
the expectations for the next generations in terms of relevance and the acquisition of scientific literacy for a future 
sustainable society. 


Literature review 
School Science Curriculum and Science Education Relevance 


Nowotny et al. (2018) have expected schools to play an important role in preparing a future sustainable society 
through their curriculum provision. However, for achieving curriculum expectations for a future sustainable society, 
@stergaard (2017) has pointed out that learning science needs to be mainly based on relevance for everyday life 
and built on societal situations to enable students’ awareness to be promoted through in science education. In 
fact, Eilks and Hofstein (2014), and Keller et al. (2017) saw sustainability and science education expectations to be 
strongly characterized through a focus on relevance aspects. Stuckey et al. (2013) have divided relevance within 
science education into three dimensions: 

e — Individual dimension - phenomena which support students’ interest and curiosity - with necessary 
information and skills for coping with their everyday life and in the future. 

e Societal dimension — focusing interaction between science and society, while promoting a personal 
contribution to sustainable development and being a responsible citizen. 

¢ Career dimension — preparing for further science-related academic education, or vocational training, 
and hence preparation for possible science-related careers. 

Studies by Akitsu et al. (2017), Halilovic et al. (2021), Hoque et al. (2022), Lay et al. (2013), and Wojuola and 
Alant (2019) have indicated a gap between science experienced in everyday life and the portrayal of the science 
education curriculum. For example, Akitsu et al. (2017) have predicted a low awareness between energy-related 
sustainability relevance dimensions and cognitive knowledge of the concept of energy. A relevance indication study 
by Murphy et al. (2006) has shown the impact of students’ engagement increased in physics lessons if relevance 
was added to the learning context. Furthermore, Eilks and Hofstein (2015), and Keller et al. (2017) have pointed 
out that in order to increase the relevance of the curriculum in the 21st century teaching and learning process, it 
is not just learning the theories and facts in science education that are important, but also the need to relate this 
to the society and career aspects. 


The Energy Concept in Science Education 
In science education, energy is usually defined through a dualistic approach: 
(i) As a core concept - mainly seen as the quantity defined in a system (Duit, 2014), which integrates all 


science disciplines and which occurs in the literature in several terminological variations, for example: 
a) big/core idea (Next Generation Science Standards, 2012; Opitz et al., 2015); 
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b) core concept (Aguiar et al., 2018; Park, 2019; Neumann et al., 2013); 

c) crosscutting concept (Crissman et al., 2014; Jin & Anderson, 2012; Fortus et al., 2015; Krajcik et al., 2014; 
Next Generation Science Standards, 2012); 

d) concept (Duit, 1984, 2014; Bachtold, 2018; Herrmann-Abell & DeBoer, 2018; Kesidou & Duit, 1993; Lancor, 
2014; Liu & McKeough, 2005; Nordine et al., 2010; Solbes et al., 2009; Yao et al., 2017). 

(ii) Asa literacy expectation — This seems to identify energy as an entity of nature and the role of energy 
as a function of the universe and our lives (DeWaters & Powers, 2013; Martins et al., 2020; USDE, 2017). 
The Energy Literacy Framework (USDE, 2017), actively seeks to promote connections across subject 
disciplines, dividing energy as: 

e Aphysical quantity that follows precise natural laws; 

e Physical processes on Earth stemming as the result of energy flow through the Earth as a system; 

¢ Biological processes depending on energy flow through the Earth as a system; 

e Various sources which can be used to power human activities, often this energy being trans- 
ferred from source to destination; 

¢ Governed by decisions which are influenced by economic, political, environmental and social 
factors; 

e That required by human society to function depending on many factors; 

e Feeding the quality of life of individuals and societies affected by their choices. 

Although there is no universal definition of energy in science education, four main characteristics can be 
identified through which the determination of the teaching and learning content within science education can 
be determined (Nordine et al., 2010; Herrmann-Abell & DeBoer, 2018; USDE, 2017). These characteristics specify 
energy as: (i) a source-form, (ii) transferable or undergoing transformation, (iii) degradable or capable of dissipa- 
tion, and (iv) being conservable. 

Research has shown that students have problems understanding the energy concept (Ben-Zvi et al., 1993; 
Driver et al., 2014; Herrmann-Abell & DeBoer, 2018; Mamlok-Naaman & Mandler, 2020). Furthermore, Cooper and 
Klymkovsky (2013), and Dreyfus et al. (2012) indicate that the fragmentation of the energy concept between sci- 
ence subjects leads to the possibility that students obtain an incomplete understanding of the energy concept as 
a whole. Furthermore, Barrow and Morrisey (1989), DeWaters and Powers (2013), and Suryana et al. (2019) have 
indicated that students tend to lack cognitive knowledge of the learning process, which needs to be overcome to 
promote energy concept. 


Structure of Learning Outcomes 


Since the turn of the millennium, schools have sought to change the intended curriculum parameters to 
outcome-based learning (Kennedy, 2006). The intended learning outcomes tend to be written in three different 
domains: cognitive, affective, and psychomotor (S6nmez, 2017). The cognitive domain is related to thinking (Bloom 
et al., 1956; Anderson & Krathwohl, 2001; Biggs & Collis, 2014), the psychomotor to hands-on activity (Simpson, 
1972) while the affective domain relates to emotions (Klopfer, 1976). 

Brooks et al. (2014) and Stalne et al. (2016) have indicated that a learning outcome (LO) is a statement that 
describes the learning that a student is expected to achieve, while other researchers see LOs as expressing the key 
learning through the gaining of knowledge, skills, or attitudes (Biggs & Tang, 2011; Elmas et al., 2020; Kennedy, 
2006; Popenici & Millar, 2015). According to Elmas et al. (2020), the more detailed the curriculum-stated LOs are, 
the easier it is to implement, assess and evaluate, plus determine its impact on educational progress. 

In stating a LO, the two main components are seen as: (i) a verb and (ii) a noun. While the verb refers to what 
a student is expected to be able to do (for example name, give examples) and the noun relates to that expected to 
be attained (for example — renewable and non-renewable natural resources) (Adelman, 2015; Anderson & Krathwohl, 
2001; Biggs & Tang, 2014; Krathwohl, 2002). 

Researchers have proposed various taxonomies in order to achieve educational goals as expressed through 
LOs, the two most commonly used for designing curriculum LOs being (i) Bloom et al. (1956) — Taxonomy of Educa- 
tional Objectives (TEO) and its revision by Anderson and Krathwohl (2001) named as Revision of Bloom’s taxonomy 
(RBT), and (ii) Biggs and Collis (1982) — SOLO (Structure of observed learning outcome) taxonomy (Chan et al., 2002; 
Whalley et al., 2006; Weay et al., 2016). 

According to Stanny (2016) the first two RBT levels (remember and understand) are considered as lower-level 
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with higher cognitive domain levels as apply, analyze, evaluate, and create. On the other hand, Chan et al. (2002) 
classify unistructural and multistructural levels, in the SOLO taxonomy, as cognitively lower levels and relational 
and extended abstract as cognitively higher levels. 

In promoting SOLO, Biggs and Tang (2011) have stated that neither TEO, nor RBT are based on the research 
of students’ learning, but rather on the opinions of educational leaders, and that the taxonomies are not actually 
hierarchical. The SOLO taxonomy is based on two main theories: constructivism (Piaget, 1954) and phenomenog- 
raphy (Marton, 1981) and the Estonian school teaching approach and the student-centered learning process are 
based on constructivism (NCBC, 2014), it is seen as appropriate to base further analysis of the NCBCSS (Estonian 
National Curriculum for Basic School, Natural Science Syllabus) on the SOLO taxonomy. 


The Estonian Science Curriculum 


The Estonian national curriculum for compulsory school (1-9" grade) is described as competence-based and 
explicitly states as one objective the need to promote scientific literacy (Soobard et al., 2015). This compulsory 
component consists of a general part plus through appendices establishes various subject curricula and descriptions 
of cross-curricular topics (NCBC, 2014). One compulsory learning area at the 7-9" grade level is stated as science, 
although actually this comprising of five sub-divisions, or subjects: (i) biology; (ii) chemistry; (iii) earth science, (iv) 
physics, and as a pre-course for future chemistry and physics studies (v) interdisciplinary science (NCBC, 2014). 

The weekly number of lessons for compulsory science subjects differ in the 7" grade, the number of science 
lessons consists of - 1 biology, 1 earth science, and 2 interdisciplinary science lessons, while in the 8" and 9" grade, 
it is 2 biology, 2 chemistry, 2 earth science, and 2 physics lessons. One lesson at the Estonian basic school level 
usually lasts 45 minutes (NCBC, 2014). 

The science 7-9" grade curricula (NCBCSS, 2014), specify: (i) general principles (e.g., competencies to be 
achieved through science learning and options for implementing cross-curricular topics), and (ii) the physical 
learning environment with the stated aim of enhancing scientific and technological literacy by putting forward: 
(a) learning and educational objectives for each of the 7-9" grade of studies, and (b) a list of topics to be taught 
for each of the five subjects together with specified learning outcomes. 

Within this frame, each subject sub-division identifies: (i) the learning content, (ii) keywords/concepts, (iii) 
recommended practical work and Information and Communication Technology (ICT) inclusions, plus (iv) intended 
learning outcomes. 

For example, renewable energy within the topic of Estonian Natural Resources in the Earth Science curriculum: 

Content: 

e “renewable and non-renewable natural resources; natural resources as sources of energy; Estonian 
minerals, mining and use”; 

e Keywords/concepts: 

e “renewable and non-renewable natural resources, underground mining, energy”; 

e Practical work and use of ICT: 

e¢ “compiling an overview of the use of natural resources in the student’s home area”, 

e Learning outcomes: 

e “name the renewable and non-renewable natural resources in Estonia and give examples of their use; give 
examples of ways of producing and using renewable energy in your home area”(NCBCSS, 2014). 


Research Questions 


To determine the relevance of learning outcomes put forward in the various Estonian science curricula, the 
following research questions are asked: 

1) To what extent does the Estonian 7-9" grade science curriculum encompass learning outcomes covering 
the three learning domains - cognitive, affective and psychomotor? 

2) Based on the cognitive process dimension — how are the energy learning outcomes conceptualized and 
developed within the various subject provisions in the Estonian 7-9" grade science curriculum? 

3) To what extent does the Estonian 7-9" grade science curriculum encompass learning outcomes covering 
the three science education relevance dimensions — individual, societal, career? 
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Research Methodology 


Document Analysis Method 


The document analysis was undertaken by a research group consisting of 2 Tartu University educational 
science PhD students (1st and 4th year) and one PhD degree lecturer from Tartu University Chemistry Institute 
(more than 10 years of higher education teaching experience). Document analysis methods are used for studying 
and analyzing the printed or electronic papers that a researcher has gathered on a certain subject (Bowen, 2009). 


Procedure 


This study was conducted based on an analysis of the current Estonian 7-9" grade science curriculum elec- 
tronic document, focusing on the identification of (i) learning outcomes lists and (ii) practical work and use of ICT 
lists (which are meant to be achieved by students in the learning process) associated with the three domains of 
learning, namely cognitive, affective, and psychomotor. Using SOLO taxonomy, a list of learning outcomes and 
practical work and use of ICT (LO) within the cognitive domain for the 7-9" grade science curricula document (in 
action since 2014 in Estonia) were identified in subjects: chemistry (C), physics (P), biology (B), earth science (ES), 
interdisciplinary science (IS). In addition, energy concept components (E, =a source-form, E,= transferable or un- 
dergoing transformation, E,= degradable or capable of dissipation, E,= being conservable) identification based on 
3 sources: (i) Herrmann-Abell and DeBoer (2018), (ii) DeWaters and Powers (2013) - a framework for energy literacy, 
(iii) US Energy Literacy Framework (2017), were taken into account. Also, science education relevance dimensions 
(R, = individual dimension, R,= societal dimension, R, = career dimension) based on Stuckey et al. (2013) paper, 
were taken into account for identifying LO relevance relatedness. 

The analysis was undertaken, based on the procedure by Nasstr6m (2009), Wei and Ou (2019), and Yaz and 
Kurnaz (2020) on the Estonian 7-9" grade science curriculum LOs stated in the Estonian national curriculum for 
basic schools. 


Procedure Steps 


First, the domain of learning outcomes based on verbal expression was determined. This was undertaken by 
identifying the verbal expression in LO for example: 

“Associate forms of life with different groups of organisms” (the cognitive domain — associate). 

Based on Hoque (2016), the cognitive domain primarily involves learning abilities which are related to mental 
or thinking processes. 

“Make wet preparations and use a light microscope to study them” (the psychomotor domain - make, use). 
Based on Hoque (2016), the psychomotor domain targets distinct physical functions, automatic responses, and 
interpretive motions. 

“Value scientific methods when drawing trustworthy conclusions” (the affective domain — value). Based on Hoque 
(2016), the affective domain is related to our feelings, emotions, and beliefs). 

For curriculum cognitive learning outcomes, the analysis used SOLO taxonomy, which classified learning 
outcomes in terms of their structural quality, which according to (Biggs and Tang, 2011) makes them useful for 
defining the levels of understanding. Similar to Soobard and Rannikmae (2015), this study did not utilize the first 
SOLO taxonomy level, since the research did not assess the item responses. 

In order to classify verbs according to the cognitive domain, the noun of within the LO sentence was taken 
into account identified: 

“Associate forms of life with different groups of organisms” (associate — the SOLO level 3 - relational). 

If several verbs appeared in one noun expression, both verbs were included in the analysis of the LO: 

“Compile and analyze schemes of the source material of photosynthesis” — compile, analyze. 

If more than one noun expression was related to the verb, this was still taken as a one verb LO: 

“Analyze the dependence of the structure of the organs of flowering plants on their tasks, location of growth and 
the means of reproduction and distribution” — analyze. 
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Analysis 


The following six steps were used to perform the analysis: (i) First, the verb(s) in the LO statement were de- 
termined; (ii) The placement of the verb in the learning domain was determined; (iii) When LO verb belonged to a 
cognitive domain, the SOLO taxonomy level was determined; (iv) LO statement relevance dimension was determined; 
(v) LO statement was compared with energy concept categories; (vi) If LO was related with energy, the association 
with four energy concept components was determined, and (vii) in order to check the reliability and validity of the 
coding process the entire 7-9" grade science curriculum was examined first three times within 2-months intervals. 


Validity 


The validity of the study was established by consulting with the former official Estonian 7-9" grade science 
curriculum developmental manager who confirmed the theoretical framework of Estonian 7-9" grade science 
curriculum LO alignment with this study and other 3 educational experts. 


Reliability 


To check reliability, active science teachers were used as the outside coders (one in each discipline: chemistry, 
physics, biology, earth science, interdisciplinary science). The results between the coders were similar. The outside 
coder results were not considered as agreement if at least one dissonance appeared between any of four coding 
categories: a) learning domain, b) SOLO taxonomy level, c) energy concept component, and d) relevance domain. 
Coder inter-rater agreement percentages reliability was determined with a summative value .88 (interdisciplinary 
science = .75, biology = .86, earth science = .89, physics = .93, and chemistry = .95). 


Research Results 
An analysis of the current Estonian national 7-9" grade science curriculum identified, in total, 782 LOs. The 


learning outcomes related to the cognitive domain (73.4%), followed by the psychomotor (22.5%) and the affective 
domains (4.1%). The distribution of LOs within three learning domains is presented in Table 1. 


Table 1 
The NCBCSS Learning Outcome Overall and the Energy Concept Learning Outcome Distribution, Based on the Three Domains 
of Learning 
Science Curriculum LO Distribution Energy Concept LOs 
Learning Domain 
Is, B, ES, P, Cc, TT, ‘JS, B, ES, P CGC, fF, 
Cognitive 32 158 137 142 105 574 13 26 47 95 27 ~—-208 
Psychomotor 35 12 41 39 49 176 20 1 7 18 16 62 
Affective 2 23 5 - 2 32 - - 3 - 1 4 
Number of LOs per Subject 69 193 183 181 156 782 3327 57 113 44 274 
Number of Lessons in 7-9" Grade 70 175 175 140 140 
Average LOs per One Lesson 1.0 1.1 1.0 1.3 1.1 


Note. |S, = interdisciplinary science; B, = biology; ES, = earth science; P, = physics; C, = chemistry; T, = total 


As shown in Table 1, the greatest number, and the percentage of cognitive domain LOs were stated in the 
subject biology (158; 81.9%), while the largest number of LOs per lesson appeared in physics (1.3). The current 
analyzing method did not identify any affective domain LO for physics. 

With the exception of interdisciplinary science, all other science subjects (biology, earth science, physics, and 
chemistry) had more than 67% of stated LOs in the cognitive domain. 

From the entire 782 LOs within the NCBCSS, the analysis revealed: 
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e 274 (35.0%) related to the energy concept LOs (Table 1), which were divided between the three learn- 


ing domains, 


e — Similar to the analysis of the entire curriculum, LOs related to energy concepts, for all subjects except 

interdisciplinary science, appeared mostly in the cognitive domain, 
e For interdisciplinary science, more than half of the LOs were categorized in the psychomotor domain, 
e As it appeared in the general analysis of LOs, biology had the largest number of energy concept LOs 


in the affective domain, 


e¢ The comparison with the total curriculum LO and the energy concept LOs showed the highest percent- 
age of energy concept LOs in interdisciplinary science. 


Table 2 


The NCBCSS Subject Cognitive Domain LO Distribution Based on the SOLO Taxonomy Levels 


Cognitive LO Distribution Energy Concept LOs 
SOLO taxonomy level 

IS, B, ES, P, C, T, Is; B, ES P, C, T, 
Unistructural 3 1 8 14 7 33 - 5 7 1 13 
Multistructural 10 43 55 82 35 225 3 1 21 56 11 92 
Relational 11 107 63 38 57 276 7 24 14 26 11 82 
Extended Abstract 8 7 11 8 6 40 3 1 7 6 4 21 
Number of Subject LOs 32 158 137 142 105 574 13 26 47 95 27 208 


Note. |S, = interdisciplinary science; B, = biology; ES, = earth science; P, = physics; C, = chemistry; T, = total 


Table 2 shows that out of 574 cognitive domain LOs, the majority (87.3%) of the LOs were identified at the 
multi-structural and relational SOLO taxonomy levels. On the lower unistructural level, 33 LOs were identified while 
on the extended abstract level, 40 LOs were identified. Furthermore, for the first two SOLO taxonomy levels (uni- 
structural and multi-structural - taken as lower cognitive process levels), table 2 reveals the following proportion 
of the entire curriculum - 258 LOs (44.9%) and at the higher levels — relational and the extended abstract — 316 
LOs (55.1%). The current LO analyzing method was able to determine only one LOs in biology at the unistructural 


level in the NCBCSS. 


The study indicated 208 (39%) energy concept LOs (see Table 2) out of a total of 534 cognitive LOs. The major- 
ity of the energy concept LOs appeared in physics (95; 45.7%), followed by earth science, chemistry, biology and 
interdisciplinary science. This study was not able to detect any SOLO taxonomy unistructural level energy concept 
LOs within interdisciplinary science and biology. The largest number of SOLO taxonomy extended abstract level 


energy concept, cognitive LOs appeared, compared with other subjects, in earth science (7). 


Table 3 


The NCBCSS Energy Concept Learning Outcome Distribution Based on the Four Energy Categories 


Science Subject 


Interdisciplinary Science 4 16 1 


Biology - 22 - 
Earth Science 12 24 4 
Physics 10 51 3 
Chemistry 7 10 1 

Total 33-123 9 


1 
14 


32 


Four Energy Categories 


2 
3 


1 
3 


9 . 
6 
9 6 


3 


4 


1,2,3,4 


19 


Note. E,=Energy source/form; E,=Energy transfer/transformation; E,=Energy degradation/dissipation, E,=Energy conservation 
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Table 3 shows the NCBCSS energy concept learning distribution, based on the four energy categories: (i) energy 
source/form, (ii) energy transfer/transformation, (iii) energy degradation/dissipation, and (iv) energy conservation. 
In total, 95 energy concept LOs (34.7%) were overlapping at least in two different energy concept categories. For 
example, the LO statement — ‘know ways of saving energy and value sustainable use of energy’ - was categorized in 
both a) energy degradation/dissipation, and b) energy conservation categories. 

The highest percentage of energy section LOs were identified in the energy transfer/transformation category 
(123; 44.9%). Table 3 further showed that energy source/form and transfer/transform categories, including their 
overlapping (table 3, E, + E, + E, ,), made up 68.6% (188) of the total energy concept LO distribution. 


Table 4 
The NCBCSS Energy Concept Learning Outcome Distribution Based on the Three Science Education Relevance Dimensions 


Put Forward by Stuckey et al. (2013) 


Curriculum LO relevance distribution Energy concept LO relevance distribution 
Subject 
R, R, R, Ri, Ris R,, Rios R, R, R, Ri, Ri; Ro; Rios 

Interdisciplinary Science 46 1 - 13 - - 9 15 1 - 9 - - 8 
Biology 152 30 3 6 2 - 24 2 - - 1 - 
Earth Science 123 41 4 9 4 2 23 23 2 6 - 2 1 
Physics 166 3 1 11 - - - 91 12 1 9 - - 
Chemistry 123 16 - 3 - 6 7 24 13 - 1 - 3 3 

Total 610 91 8 42 1 12 18 177 51 3 25 1 5 12 


Note. R, = Individual dimension; R, = Societal dimension; R, = Career dimension 


Table 4 gives an overview of the Estonian 7-9" grade science curriculum and energy concept LO distribution, 
based on science education relevance dimensions (Stuckey et al., 2013). 

The table shows mainly science curriculum LO were identified in the individual dimension (R,=610, 78.0%) 
although, in the same dimension, the proportion appearing in the energy concept was 177 (64.6%). Compared 
with the entire 7-9" grade science curriculum relevance distribution, the LOs in the societal dimension (R,) are 
appearing with a higher percentage in the energy concept LO relevance distribution. 

Similarly, to energy concept categories distribution, also in the science education relevance dimension ap- 
peared to have an overlapping effect between three dimensions. For example, the LO statement - ‘understand the 
importance of natural sciences and technology in everyday life’ - was categorized in all three dimensions. 

Taking into consideration a single distribution and overlapping appearing in science education career dimen- 
sion (table 4, R, + Reg + Reg + Rigs) leads to the result for the entire science curriculum with 39 LOs (5.0%) and in 
the energy concept 21 LOs (7.6%). A similar calculation procedure in the societal dimension gives for the entire 
curriculum LOs distribution a result of 20.8% and for the energy concept LOs 33.9%. 


Discussion 


The purpose of this study was to determine: a) the distribution of learning outcomes (LOs) in the NCBCSS, 
based on three domains of learning, namely, cognitive, psychomotor, and affective domains, b) how the crosscutting 
energy concept is emphasized in comparison with the entire science curriculum, and c) identifying the common 
relevance dimension of the energy concept LOs. 


Research Question 1 
This research identified 782 LOs (Table 1) in the NCBCSS and LOs were distributed across 3 domains (cognitive, 
psychomotor and affective), which is to be expected. In general, the current Estonian 7-9" grade science curriculum 


is strongly biased to the cognitive domain side and the excessive proportion of cognitive domain LOs (73.4%) in 
the curriculum may give, according to Elmas et al. (2020), a potential focus for a teacher-centered learning ap- 
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proach. In contrast to the study by Kacovsky et al. (2021), which stated that no major differences existed between 
subjects in the Czech, Polish, Estonian, and Slovenian science curricula, this research confirmed that major cogni- 
tive LO differences existed between different science subjects, also in the LO distribution and between the three 
LO domains. The difference between this study and the one by Kacovsky et al. (2021) could be explained by the 
use of slightly different methodologies used for the analysis. However, several studies (Elmas et al., 2020; Lee et al., 
2015; Wei & Ou, 2019) have found that there is no strict guidance material on how many LOs and in which learning 
domain they need to be stated in a science curriculum. Roblin et al. (2018) point out that from one side a detailed 
curriculum with a large number of LOs allows better assessment and has a higher potential to be implemented 
in schools. But at the same time, Nodding (2013) notes that a long and detailed list of LOs can affect teachers and 
students’ negative promotion to be creative. 

The results in Table 1 show in physics zero LOs in the affective domain. This might be possibly due to the spe- 
cifics of the subject physics which emphasizes a large number of cognitive learning outcomes, which do not allow 
the incorporation of values in the physics teaching process (Keller et al., 2017). This is seen as a matter of concern 
worth further discussion, especially since Holbrook and Rannikméae (2009) point out that values are important factors 
in shaping scientific literacy and the Estonian Education Strategy for 2021-2035 (EMER, 2020) indicates a need to 
increase learner's values through science education and has also stated the challenge for current subject-centered 
learning in general education, which would further support students’ development for a future sustainable society. 

Results in Table 1 indicate an almost 2:1 LO ratio between the cognitive and psychomotor domains in chemistry, 
pointing to the importance of the teacher involving students in carrying out laboratory and practical assignments, 
thereby supporting the cognitive domain (Seery, 2019). Nevertheless, situations can arise whereby it is less plausible 
to perform a given laboratory or practical activity perhaps because of teachers’ beliefs (Vaino et al., 2013). In such 
cases, the cognitive domains LOs become more relevant, even though the diversity of teaching methods and activi- 
ties are critical components for enhancing scientific literacy (Millar, 2006; Dragos & Mih, 2015). In fact, diversity can 
be a critical factor to take into account, since unvaried chemistry learning processes can cause a decline in student 
interest for the subject (Ferrell & Barbera, 2015; Vaino et al., 2013). This is seen as a matter of concern of worthy 
further discussion and research to determine the actual reality of chemistry lessons in the Estonian school context. 

Table 2 shows an almost equal cognitive LO distribution between low- and high-level SOLO taxonomy levels 
within 5 science subjects. However, there are anomalies e.g. subject physics compared with chemistry and biology. 
Table 2 shows that the subjects, physics and chemistry have the same number of study lessons in the 8" and 9" 
grades. At the same time, there are more LOs in physics and the cognitive percentage of LO in physics is greater at 
the lower SOLO levels, while in biology only one SOLO taxonomy lower level LO. Elmas et al. (2020) have indicated 
that lower level LOs in the curriculum may be embedded within the higher level LOs. Westbury (2016) has stated 
that the unequal cognitive LO distribution between science subjects can be explained due to the curriculum de- 
velopment process when the specialists do not interrelate with the work of others, which can cause developing 
differences between variables in the curriculum. However, Biggs and Tang (2011) have pointed out the importance 
of LO quality, rather than quantity, since a large number of low-level LOs can be set seeking equivalence with a 
low number of high-level LOs and this can lead to the high average LO numbers per lesson — 1.3 (Table 1) - in 
the Estonian 8-9" grade physics lessons. The results from this study suggest the need to pay more attention to 
the distribution of LOs especially in physics, biology, and chemistry so that the cognitive demand is more evenly 
distributed across the subjects. 


Research Question 2 


35% of all LOs in the science curriculum were classified as related to the energy concept LOs (table 2). Hence 
this study confirms that energy is an important concept in science education. The current study showed similari- 
ties with Duit (2014) finding, which stated that more energy topics are discussed in the subject of physics. This 
is explained by the multiple conceptual topics covered in physics education, for example, kinetic and potential 
energy, heat, mechanical work, electrical energy, and power (Duit, 2014). 

This study revealed that 7"-grade curriculum subject interdisciplinary science is portrayed as an important 
starting point for understanding energy concepts since nearly 50% of the LOs in this subject were identified as 
relating to the concept of energy (table 2). This result may give science teachers a possible indicator for restructur- 
ing the planned activities in the teaching process and science teachers may build a new teaching strategy through 
the energy concept. 
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Studies indicated that energy fragmentation across science subjects can lead to poor conceptualization of the 
energy concept (Eisenkraft et al., 2014; Stubbs, 1985). This suggests curriculum designers can consider the creation of 
a separate subject thus avoiding energy concept fragmentation. An integrated approach to the understanding of the 
energy concept may also help to achieve better scientific literacy (Duit, 2014). Of course, the current process can be 
also undertaken by science subject teachers, since current Estonian educational legislation does not restrict schools 
from integrating subjects independently, if the LOs in the curricula are covered by the end of the 9" grade. Based on 
the results of this study, further research is needed to determine to what extent students have achieved the energy 
concept LOs and whether or not fragmentation between subjects limits success in understanding the energy concept. 


Research Question 3 


The majority of the energy concept LOs were tilted in the source/form and transfer/transform category direc- 
tion (table 3). These current results need further research since studies made abroad — Ben-Zvi et al. (1993), Driver 
et al. (2014), Herrmann-Abell and DeBoer (2018) have shown high satisfactory results on energy source-form and 
transfer-transform categories. At the same time, in the previously mentioned studies dissipation-degradation and 
conservation categories had an increased proportion of energy concept misconceptions. It might be possible that 
the Estonian 7-9" grade science curriculum’s overwhelming focus on one energy category can be a hindering fac- 
tor for fully understanding the energy concept. Overall, in the energy production scale, energy conservation and 
dissipation-degradation are the main causes of global warming (Bilgen & Sarikaya, 2018). 

The lack of similar studies doesn’t allow high-level argumentation about the emphasis on energy conceptualiza- 
tions in science education seen as relevant at individual and social dimensions. At the same time, it is also important 
to indicate that without relevant conceptualizations of energy or science education in general, it is difficult to make 
conclusions about interactions between humans and society, and hence the place of science education (Stuckey et 
al., 2013). 

The results from Table 4 indicate a low number of LOs in the science subjects, which do not relate well to promot- 
ing career orientations in the Estonian science curriculum, which forces the promotion of the science career process 
mainly to rest on Estonian science teachers’ personal contribution. Similar results appeared also in research by Akitsu 
et al. (2017), which indicated a gap between energy education needs and curriculum possibilities. The study by Kotkas 
et al. (2021) in the Estonian context has already revealed both teachers and students’ low experiences in the science 
career field, although a deeper analysis with the NCBCSS was not made. This topic needs also to be considered by 
curriculum designers since Orthner et al. (2013) and Shin et al. (2015) have shown through their research a positive 
effect on students’ values of science-related professions if career-related topics are addressed in science lessons. 


Conclusion and Implications 


The current study indicated that in the Estonian 7-9" grade science curriculum LOs are stated in all three learning 
domains (cognitive, psychomotor, and affective) but mainly identified in the cognitive domain. Nevertheless, learning 
domain distribution between the different science subjects is seen as uneven and no affective domain LOs are given 
in the physics curriculum for grades 8-9". 

Within the Estonian 7-9" grade science curriculum cognitive LOs process dimension there is an almost equal 
distribution between low- and high-level cognitive learning outcomes. The majority of LOs in the entire 7-9" grade 
science curriculum and also in energy concepts relate to SOLO taxonomy at multistructural and relational levels. 

The energy concept transfer and transform category dominance among four energy categories in the Estonian 
7-9th grade science curriculum can give science teachers a first indicator of how to start solving energy concept 
fragmentation between different science subjects. The results showed that in the Estonian 7-9" grade science cur- 
riculum LOs distribution appeared a lack in dealing with important aspects of the energy concept (such as energy 
conservation and dissipation/degradation) and LOs in the science subjects do not relate well to promoting careers 
in the science field. 

The above-stated conclusions can be the base for raising awareness, opening discussions in applicable countries 
where 7-9" grade science education is taught as a separate discipline. This analysis of the Estonian lower second- 
ary (7-9 grade) science curriculum, has extended our knowledge of the importance of crosscutting concepts in 
the curriculum, so as to establish a constant focus in the teaching/learning process in science education relevance 
perspective. Moreover, the value of the method used for energy concept analysis in the current study can be applied 
internationally to other educational disciplines for curriculum development. 
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Limitations 


One of the current study limitations is the lack of an analysis of the 1-6" science curriculum, which may impact 
LOs for 7-9" grade study. Also, a limitation is the verb-centered approach to identify LOs, which can affect the results 
and with that in mind, the list of nouns defined in this study with one single verb, which can underestimate the extent 
of the learning outcome content, and at the same time, overestimate the LO statement quality, when two or more 
verbs appear in one learning outcome statement. Also, in the Estonian 7-9" grade science curriculum, there can be 
unclassified energy concept LOs, which are not explicitly linked to the energy concept LOs, e.g., gravitational forces 
addressed as pressure. 
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Abstract. Physics education aims to 
develop sophisticated thinking abilities and 
enhance conceptual depth, but student 
interest has decreased, emphasizing the 
importance of teacher quality in fostering 
success and positive attitudes. The aim 

of the study was to examine the physics 
teachers’ self-efficacy, innovative teaching, 
and comprehensive teaching approach 
levels. As data collecting instruments, 
self-efficacy, innovative physics teaching, 
and comprehensive teaching approach 
scores were utilized. In the pilot research 
group, validity and reliability assessments 
were conducted on the scales. The primary 
research involved 241 physics teachers. As 
the measurement variables were constantly 
changeable, Latent Profile Analysis was 
utilized. The present study’s results suggest 
that teachers have high self-perceptions 

in various dimensions related to teaching 
effectiveness, including self-efficacy, 
innovative teaching, and comprehensive 
teaching approach. These results are 
consistent with previous research that has 
shown a positive relationship between 
teacher self-perceptions and teaching 
effectiveness. Future research could 
explore the relationship between teacher 
self-perceptions and student achievement 
to better understand the impact of 
self-perceptions on student outcomes. 
Overall, the results of this study highlight 
the importance of supporting teachers’ 
self-efficacy and providing opportunities 
for innovative teaching and professional 
development to improve comprehensive 
teaching approach. 

Keywords: physics teachers, self-efficacy, 
innovative teaching, comprehensive 
teaching approach, Latent Profile Analysis 
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Introduction 


Physics education in lower and upper-secondary schools has been 
identified as a challenging area due to the abstract and complex nature of 
the subject matter (Duda & Garrett, 2008; Ekici, 2016). The purpose of phys- 
ics education in the twenty-first century is to build sophisticated thinking 
abilities and enhance conceptual depth (Bao & Koenig, 2019). Research has 
indicated a decline in students’ interest in the field, highlighting the need 
for effective teaching strategies to foster engagement and understanding 
(McDonald & Waite, 2019; Ozis et al., 2018; Veloo et al., 2015). As with all 
other courses, teachers play a crucial role in enhancing student success and 
attitudes in physics sessions. 

More emphasis has been placed on the relationship between teacher 
quality and student accomplishment. Some research indicates that more 
qualified teachers may have an impact on student learning in the classroom 
(Clark et al., 2021; Darling-Hammond, 2000). As the understanding of the 
complex nature of student learning and teaching techniques has changed, 
so has the area of research on teacher efficacy. Recently, a multilayered 
approach has arisen that captures the convergence of numerous factors 
and settings on how teachers define teaching and ‘practice’ teaching in the 
classroom (Devine et al., 2013; Watt & Richardson, 2008). 

Teacher quality, including teaching competence, is a significant compo- 
nent of teacher training. Numerous training practices and implementations 
occur in both pre-service teacher training and professional development. 
Teacher quality, particularly teaching competence, is linked to numerous 
variables such as teacher self-efficacy (Romel et al., 2021), teaching quality 
(Kennedy, 2006), subject expertise (Loewenberg Ball et al., 2008), peda- 
gogical competence (Snoek, 2021), pedagogical content knowledge (Park 
et al., 2020), and technological pedagogical content knowledge (Karakaya 
& Avgin, 2016). 

As aresult of the school reforms taking place in China, learner-centered 
physics teaching is anticipated. In China, all lower-secondary and upper- 
secondary school students are required to study physics, a subject that is 
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regarded as vital (Zhang et al., 2020). In this regard, research is conducted to enhance the expertise of physics 
teachers in this subject (Fu & Clarke, 2019). According to the results of the bibliometric analysis of physics teach- 
ers in China, the main research hotspots of physics teaching are teaching, physics teaching aids, and educational 
evaluation (Wu et al., 2021). According to both studies, the teaching dimension is an essential idea for both 
researchers and educators. To organize relevant research, it is vital to determine the qualifications and level of 
teachers in this subject. 

Self-efficacy, or belief in one’s ability to complete a particular task, has been demonstrated to be positively 
associated with learning and motivation across domains (Bandura, 1997). According to Pajares (2003), self -ef- 
ficacy is person's confidence in their ability to accomplish their goal. When these people become teachers, we 
can think of them as the belief that they have the necessary qualifications and skills to reach the goals they have 
set about teaching. 

It is believed that educators who have strong ideas in their own competence put in more effort, participate 
more actively in informal learning activities, are more persistent, and experience less stress (Bandura, 1997; Lohman, 
2006). Furthermore, the self-efficacy beliefs of teachers have been shown to be connected with the pedagogical 
practices of such teachers (Tschannen-Moran et al., 1998). Teachers who have high levels of self-efficacy are more 
likely to demonstrate effective classroom management (Woolfolk et al., 1990), use more innovative teaching 
methods (Ghaith & Yaghi, 1997; Guskey, 1988), set higher learning goals for their students (Ross & Bruce, 2007; 
Wolters & Daugherty, 2007), and encourage student autonomy (Woolfolk et al., 1990). 

According to Zimmerman (2000), in terms of its substance, self-efficacy assessments place greater emphasis 
on performance capacities than on physical or psychological features. Self-efficacy has a multiple structure rather 
than a unique structure. Although there is a self-efficacy belief about teaching in general, there are self-efficacy 
belief scales that contain different aspects of teaching. For example, Haryanto et al. (2023) measured self-efficacy 
by associating it with the pedagogical content knowledge framework. In a research done by Ogan-Bekiroglu and 
Aydeniz (2013) on pre-service physics teachers, the self-efficacy of pre-service physics teachers at several phases 
of the teaching process, including planning, implementation, assessment, and general teaching, was measured. 
In the study conducted by Lekhu (2016), the self-efficacy of physics teachers in performing laboratory experi- 
ments was measured. As a result, data collection tools should be used to measure self-efficacy beliefs that include 
qualities suitable for research purposes. It is not possible to use a generally accepted measurement tool that is 
suitable for every teaching field. 

The teaching strategy is the general application of teacher-determined methodologies to achieve learning 
objectives. When we think about it in the context of physics lessons, for example, it can be thought of as helping 
students to structure the concepts related to physics correctly in their minds (Grayson, 2004). According to Ukala 
(2018), teaching that is innovative takes a proactive approach to integrating new teaching ideas and methods 
into the classroom setting, where they may be used to benefit both the teaching and the learning that takes 
place there. Teachers are constantly being challenged to improve their own ways of thinking and the teaching 
strategies they use in the classroom. This is done with the goal of enhancing students’ learning attitudes and 
abilities, thereby preparing them to meet the challenges posed by an increasingly complex and dynamic world 
(Gul & Rafique, 2017). 

Many teaching strategies such as multiple intelligence teaching strategy (Gul & Rafique, 2017), project- 
based teaching (Siew et al., 2015), experiential teaching strategy (Bada & Akinbobola, 2017), reciprocal teaching 
strategy (Mafarja et al., 2022), and real problem solving (Lopez-Jimenez et al., 2021) have been applied in the field 
of physics education. “Innovative” is defined in the dictionary as featuring new methods; advanced and original 
(dictionary.cambridge.org). The innovative teaching strategy, on the other hand, can be defined as strategies 
that are out of the traditional teaching strategies and offer an innovative perspective to achieve teaching goals. 

The creation of teaching content is an extremely important factor in the improvement of academic perfor- 
mance of students across all subject areas (Rios et al., 2018). The degree to which a teaching strategy or method 
supports learning outcomes is directly proportional to the success of the approach or technique. If the students’ 
learning is adequately supported by the teaching methods utilized, it can be concluded that those teaching 
methods are effective. In the study done by Dwikoranto et al. (2020), a project-based laboratory learning strategy 
was utilized to enhance students’ science process skills and levels of creativity. Because of the study, it is possible 
to conclude that the teaching employed is effective, as the anticipated growth in students’ creativity and science 
process skills is realized. Effectiveness is not only determined by the teacher. The fact that students interpret the 
lesson’s materials and approaches effectively is a success-enhancing aspect. In the study done by Klein et al. 
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(2021), the efficiency of the university-level teaching delivered throughout Europe during the pandemic era was 
evaluated from the perspective of the students. 

Latent profile analysis (LPA) is a categorical latent variable method that identifies latent subpopulations 
within a population based on a particular set of characteristics. LPA thus presupposes that individuals may be 
categorized with variable degrees of probability according to their personal and/or environmental characteristics 
(Spurk et al., 2020). The objective of all mixture models is to give a probabilistic categorization of individuals into 
latent classes based on a statistical model, where each class is one of K discrete latent variable categories c. In 
contrast to conventional component analysis or IRT modeling, the latent variable is hence categorical as opposed 
to continuous (Bauer, 2022). Using the means of continuous observable variables, latent profile analysis attempts 
to recover hidden groupings (Oberski, 2016). In many educational research studies, latent profile analysis is em- 
ployed. For instance, research has been undertaken on teacher burnout during the COVID19 epidemic (Raducu 
& Stanculescu, 2022), teacher attitudes and practice while using technology in the classroom (Thurm, 2018), and 
the identification of future secondary school teachers (van Rooij et al., 2020). 

This study aimed to explore teachers’ self-efficacy, innovative teaching, and comprehensive teaching ap- 
proach levels in the context of physics education, with a particular focus on the Chinese educational system, 
where physics is considered as a vital subject (Fu & Clarke, 2019; Wu et al., 2021; Zhang et al., 2020). To address 
this aim, the following research questions were posed: 

¢ What are the levels of teachers in terms of self-efficacy, innovative teaching, and comprehensive 
teaching approach? 

e¢ What are the correlations among sub-dimensions of self-efficacy, innovative teaching, and compre- 
hensive teaching approach? 

e¢ =Whatare the latent profiles of teachers in terms of self-efficacy, innovative teaching, and comprehen- 
sive teaching approach? 


Research Methodology 


The study employed a quantitative approach, conducting exploratory analyses for data interpretation. The 
measurement tools were directly applied to the sample population for data collection. The research was initiated 
at the beginning of the 2022-2023 academic year. 


Sample 


The research involved two separate samples: one for a pilot study and the other for the main study. The 
pilot study engaged 120 teachers who completed questionnaires. Of these, 107 participants provided complete 
responses, which were subsequently utilized. The pilot study aimed to secure a minimum sample size of 100 
participants, as suggested by Hair et al. (2010). Notably, no demographic data related to the participants was 
collected during the validity and reliability assessment of the scale. Following the completion of the validity and 
reliability processes for the scales in the pilot study, the same tools were administered to the main study sample. 
This sample consisted of 320 physics teachers from Jiangsu Province. Of these teachers, 241 returned the scales 
without missing data, and their responses were utilized for the main study analysis. 


Data Collection Tools 
To acquire the scales, an extensive literature review was performed, identifying potential scale items for use 
in self-efficacy, innovative teaching, and comprehensive teaching approach within physics education domains. 
The scale items were subsequently evaluated by a panel of specialists, including two physics educators and two 
assessment and evaluation experts. A separate sample, possessing similar attributes but not included in the 
primary study, was chosen to validate and assess the reliability of the scales used in the research. 


Self-efficacy 


According to the exploratory factor analysis result, KMO value was calculated as .828 and Bartlett's Test of 
Sphericity result was calculated as (x7= 1898, df= 66, p < .01). As a result of the Exploratory Factor Analysis, item 
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9 was removed from the scale. A two-factor structure was obtained. In the first factor, there are 2nd, 5th, 10th, 
12th, 3rd, 6th items and this factor is named as “teaching self-efficacy”. The factor load is between .72 - .98. In 
the second factor, there are 8th, 4th, 1st, 7th, and 11th items. The factor load is between .99 and .72. The second 
factor is named as “affective domain “. The first dimension can explain 39.5 percent of the variance of the scale. 
The second dimension explains 32.4%. In total, 71.9 of the scale variances can be explained by these factors. As a 
result of confirmatory factor analysis; The first model calculated y7/df (417/43 = 9.69), CFI = .806, TLI = .752, SRMR 
= .093 and RMSEA = .282. Since the indices were not sufficient, the modifications suggested by the program 
were made. In the second model, x7/df (37.5/40 = .937), CFl = 1, TLI = 1, SRMR = .0615 and RMSEA =.0. In the third 
step, the reliability values were calculated. Cronbach's a = .967, McDonald's w = .969 for Teaching self-efficacy, 
Cronbach's a = .938, McDonald's w = .940 for self-efficacy to handle psychological factors, Cronbach's a = .914, 
McDonald's w = .918 for the whole scale. As a result, a valid and reliable scale was obtained to measure the self- 
efficacy of physics teachers. 


Innovative Physics Teaching 


According to the exploratory factor analysis results, the KMO value was calculated as .911 and the Bartlett’s 
Test of Sphericity result was calculated as (x? = 2329, df = 153, p< .01). As a result, a three-factor structure was 
obtained. In the first factor, there are 5th, 14th, 8th, 7th, 12th, and 9th items, and the factor is named “multiple 
teaching”. The factor load is between .64 - .93. In the second factor, there are 2nd, 3rd, 4th, 6th, 11th, and 15th 
items. The factor load is between .67 and .92. The second factor is called “self-directed learning”. In the third fac- 
tor, there are 1st, 10th, 13th, 16th, 17th, and 18th items. The factor load is between .56 and .90. The third factor is 
named “student-centered”. The first dimension can explain 25.4 percent of the variance of the scale. The second 
dimension explains 24.6% and the third dimension is 23.8%. In total, 73.8% of the scale variance can be explained 
by these structures. As a result of confirmatory factor analysis, y7/df (197/132 = 1.49), CFI = .972, TLI = .968, SRMR 
.0505, and RMSEA .0673 are calculated. No modification was applied because the indices were sufficient. In the 
third step, reliability calculations were made. Cronbach's a = .932, McDonald's w = .943 for multiple teaching, 
Cronbach's a = .935, McDonald's w = .940 for self-directed learning, Cronbach's a = .934, McDonald's w = .940 for 
students-centered, Cronbach’s a for the whole scale = .946, McDonald's w = .949. 


Comprehensive Teaching Approach 


According to the exploratory factor analysis, the KMO value was .901 and the Bartlett’s Test of Sphericity result 
was calculated as (y?= 2186, df= 190, p < .01). A two-factor structure was obtained. In the first factor, 10 items (1, 
2, 3, 6, 7,9, 10, 11, 18, 19) were collected. The factor is named “planning”. The factor load is between .53 - .96. The 
remaining 10 items (4, 5, 8, 12, 13, 14, 15, 16, 17, 20) for the second factor were collected. The factor load is between 
.59 - .98. The second factor is named “implementations”. The first dimension explains 31.5 percent of the scale 
variance. The second dimension explains 31%. In total, 62.5 percent of the scale variance can be explained. The 
first model indices formed at the end of the confirmatory factor analysis were calculated as x7/df (404/169 = 2.39), 
CFI = .892, TLI = .879, SRMR .0656, and RMSEA .113. Since the values were not local, the indices were calculated as 
y’/df (259/160 = 1.62), CFI = .955, TLI = .946, SRMR .0579 and RMSEA .0752 after the modifications suggested by 
the program were made. In the third step, reliability calculations were made. Cronbach's a = .935, McDonald's w 
= .943 for the Planning dimension, Cronbach's a = .937, McDonald's w = .943 for the implementation dimension, 
Cronbach's a = .939, McDonald’s w = .943 for the whole scale. 

The scales, whose validity and reliability processes were completed, were applied to 320 physics teachers 
working in Jiangsu Province. In the scales, 241 of them were returned completely. 


Data Analysis 
It was planned to make a classification based on the average scores obtained by the teachers in the variables. 


Latent Profile Analysis was used because the measurement variables were continuously variable. In the study, the 
snowRMM package was used in the Jamovi (2.3.21) program. 
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Research Results 


The self-perceptions of teachers were explored. Two subdimensions comprise the self-efficacy scale. In both 
subcomponents, teachers perceived themselves to be very skilled. As indicated in Table 1, the average for the 
teaching self-efficacy sub-dimension, which measures teachers’ self-efficacy in the overall teaching field, is 4.28, 
indicating that teachers viewed themselves as very competent in this sub-dimension. The mean for the sub- 
category affective domain, which teachers deemed sufficient to overcome students’ reluctance to learn about 
the emotive component, was 4.31. Comparison to all other sub - dimensions, teachers perceive themselves to 
be most competent in this one. 

The innovative physics teaching scale is composed of three dimensions. Teachers considered themselves 
adequate (x = 3.91) on the multiple teaching sub - scale, which measured the utilization of a variety of teaching 
aids and resources for learners who have different learning styles. Teachers believed they are adequate (x= 4.01) 
in guiding students toward self-directed learning and selecting the most effective teaching technique. In addi- 
tion, teachers believe themselves competent (x = 3.91) in favoring student-centered methods and tactics in the 
classroom. Thus, teachers perceived themselves to be skilled in innovative teaching. 

There are two sub-dimensions on the comprehensive teaching approach scale. In the “planning” sub- 
dimension, which measured the planning and organization of successful teaching, teachers believe they do 
adequately (x = 4.12). The second sub-dimension, “implementations,” assessed how teachers viewed their plans’ 
implementation. The mean on this dimension is 4.01, and teachers felt themselves adequate. 


Table 1 
Descriptive Statistics of Participants 


Dimensions x SD 
Teaching self-efficacy 4.28 0.511 
Affective domain 4.31 0.497 
Multiple teaching 3.91 0.868 
Self-directed learning 4.01 0.875 
Student-centered 3.91 0.910 
Planning 4.12 0.782 
Implementations 4.01 0.783 


Pearson correlation value, as shown in Table 2, was calculated for the relationship between sub-dimensions. 
The highest correlation was between self-directed learning and multiple teaching sub-dimension. The level of 
correlation between teaching self-efficacy and multiple teaching and teaching self-efficacy and self-directed 
learning were not statistically significant. Again, the relationship between implementations and multiple teaching 
and implementations and self-directed learning were not statistically significant. The level of relationship between 
other sub-dimensions was statistically significant. 


Table 2 
Correlations Among Dimensions 
Dimensions (1) (2) (3) (4) (5) (6) 


Teaching self-efficacy (1) — 
Affective domain (2) .640*** — 
Multiple teaching (3) 102 .135* _ 
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Self-directed learning (4) 


Student-centered (5) 


Planning (6) 


Implementations (7) 
Note. * p< .05, **p<.01,***p<.001 
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(1) 
095 
139° 

42g** 
a 


(2) 
2133" 
130" 

Aage 
aa 


(3) 
892"* 
769** 

156* 

057 


(4) 
796*** 
165* 
052 


(5) 


.180** 


.222*** 


(6) 


.693*** 


For each model parameterization, one, two, three, four, or five latent profiles were evaluated. To analyze the 
performance of the 15 combinations of models and the number of profiles, we conducted an analytic hierarchy 
approach based on the following five generally used information criteria: AIC, AWE, BIC, CAIC, CLC, KIC, SABIC, ICL, 
and Entropy, as shown in Table 3. A smaller number for each information criterion indicates a superior model, and 
Entropy should be close to 1. The matrix produced from the three models’ IC, AWE, BIC, CAIC, CLC, KIC, SABIC, ICL, 
and Entropy values was analyzed. The analytical hierarchy technique recommends using model 2 parameterization 
with five latent profiles (varying variances and covariances kept to zero). In addition, model 2 with five profiles got 
the lowest scores for each of the information criterion. 


Table 3 
Latent Profile Analysis 


Model Class 


1 
1 
1 


Ph MBO NM DM PD 


oOo wo WO WoO WwW 


1 


ao FF Ww PY 


Fr wo DY 


5 


LogLik 


-1853 
-1707 
-1399 
-1294 
-1219 


-1853 
-1561 
-1273 
-1127 
-1056 


-1339 
-1307 
-1196 
-1158 
-1118 


AIC 


3733 
3458 
2858 
2664 
2529 


3733 
3181 
2635 
2371 
2260 


2747 
2699 
2494 
2434 
2370 


AWE 


3899 
3719 
3215 
3117 
3078 


3899 
3526 
3160 
3076 
3144 


3164 
3212 
3103 
3138 
3170 


BIC 


3782 
3535 
2962 
2796 
2690 


3782 
3282 
2788 
2577 
2518 


2869 
2849 
2672 
2640 
2603 


CAIC 


3796 
3557 
2992 
2834 
2736 


3796 
3311 
2832 
2636 
2592 


2904 
2892 
2723 
2699 
2670 


CLC 


3707 
3416 
2800 
2590 
2439 


3707 
3125 
2549 
2255 
2114 


2679 
2615 
2394 
2318 
2238 


KIC 


3750 
3483 
2891 
2705 
2578 


3750 
3213 
2682 
2433 
2337 


2785 
2745 
2548 
2496 
2440 


SABIC 


3738 
3465 
2867 
2676 
2544 


3738 
3190 
2649 
2390 
2284 


2758 
2713 
2511 
2452 
2391 


ICL 


-3782 
-3545 
-2978 
-2810 
-2699 


-3782 
-3288 
-2803 
-2589 
-2532 


-2869 
-2860 
-2681 
-2651 
-2616 


Entropy 


0.913 
0.958 
0.955 
0.959 


Note: Model 1: Equal variances and covariances fixed to zero; Model 2: Varying variances and covariances fixed to zero; Model 3: 


Equal variances and equal covariances 


present the situation of the groups according to each variable in more detail. 


https://doi.org/10.33225/jbse/23.22.668 


According to the groups, their distribution according to each variable is seen in Figure1. In the 5th group, 
innovative physics teaching dimensions are seen to be partially low. Different visualization tools were used to 
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Figure 1 


Distribution of Variable Based on Profile 
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Based on LPA findings, as shown in Figure 2, teachers were categorized into five distinct groups. The first category 
was known as moderately competent. They perceived themselves to be average in all aspects relative to other groups. 
Moreover, all sub-dimension averages were near one another. It may be considered that the teachers in this category 
to be those who believe they can use my knowledge. There are 64 teachers in the first group, which accounts for 26.6% 
of the total. 

The second set of teachers was labeled “Low self-efficacy - moderate teaching skills.’ The judgments of self-efficacy 
of the teacher in this group were slightly above average, but only on the innovative teaching and comprehensive teach- 
ing approach scales, as revealed by assessing other teaching skills. There were 77 teachers in this category, or 32 percent 
of the total membership. 

Thirdly, teachers in the Very High Competent category believed they were proficient in all subject areas. The scale 
averages were significantly higher than those of the other categories. The number of educators in this category was 76, 
making up 31.5 percent of the total. 

Non-High self-efficacy - High teaching skills characterized the fourth group of educators. This group had the lowest 
judgments of self-efficacy compared to other groups. Nonetheless, these teachers scored exceptionally well on measures 
of creative teaching and effective education. The teachers in this category considered themselves to be humble. This 
group of twelve individuals represented 5% of the entire group. 

The fifth group of teachers had high self-efficacy but inadequate innovative teaching skills. This group of teachers 
had a high level of self-efficacy but chose a traditional approach and was deficient in innovative teaching. There were 
five teachers in this group, or 5% of the total membership. 
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Discussion 


Teachers’ self-perception is crucial to a comprehensive teaching approach and student performance (Clark 
et al., 2021). The purpose of this study was to examine educators’ self-perceptions on many dimensions, such 
as self-efficacy, innovative teaching, and comprehensive teaching approach. The focus of this work was on the 
two sub-dimensions of self-efficacy, the three dimensions of creative teaching, and the two sub-dimensions of 
comprehensive teaching approach. 

Self-efficacy is a person’s confidence in their capacity to accomplish particular activities successfully (Pajares, 
2003). In the context of teaching, self-efficacy refers to a teacher's confidence in their capacity to educate success- 
fully. According to this study, teachers perceive themselves as highly competent in both teaching self-efficacy and 
the affective domain. The emotional domain refers to a teacher’s capacity to manage the emotional components 
of education, such as addressing student misconduct and inspiring students. Findings from the present study 
indicate that teachers feel they are adept at controlling the emotional components of teaching, a crucial feature 
of good teaching. These results are consistent with other studies (Chan, 2008; Higde et al., 2014; Lekhu, 2016). 

Innovative teaching refers to techniques of teaching that are innovative and encourage active student 
learning (Ukala, 2018). Multiple teaching, self-directed learning, and student-centeredness comprised the three 
variables utilized to assess innovative teaching in this study. According to the data, teachers believe they perform 
acceptably in all three categories. Multiple teaching methods relate to the use of a variety of strategies to target 
students with distinct learning styles. 

The use of multiple representation systems by teachers also supports multiple teaching in a sense. Teachers’ 
use of multiple representation systems differs. For example, in the study conducted by Masrifah et al. (2020), it is 
stated that the proficiency of physics teachers in using multiple representations is at a moderate level. However, 
following professional development for physics teachers, these educators have started using multiple representa- 
tions more effectively (Conceicao et al., 2021). 

Self-directed learning refers to fostering student autonomy and accountability for their own education. 
Student-centeredness refers to pedagogical practices that prioritize the interests and needs of students. Obser- 
vational data from a study of physics teachers confirmed that the teacher's approach shifted from a controlling, 
teacher-dominated style to one that fostered more autonomous learning opportunities for independent student 
learning, teacher-student interactions, and student-student interactions (Zhang et al., 2020). 

A comprehensive teaching approach is the capacity of a teacher to plan and conduct successful education. 
The present study utilized a scale with two sub-dimensions to assess a comprehensive teaching approach: plan- 
ning and implementation abilities. The results indicate that teachers regard themselves as satisfactory in both 
subdimensions. Planning abilities refer to a teacher's ability to create lesson plans that are aligned with teaching 
objectives and suit the requirements of students (Van Dusen et al., 2021). Implementation abilities refer to a 
teacher's ability to effectively implement lesson plans and change education to meet the requirements of stu- 
dents (Haryanto et al., 2023). Comprehensive teaching approach is related to pedagogical content knowledge 
(Ogan-Bekiroglu & Aydeniz, 2013). According to Wang et al. (2016), as the basic education reform sweeps across 
mainland China, teachers are faced with the problem of continually enhancing their professional literacy in order 
to accomplish their jobs well. Teachers’ qualifications and abilities often increase throughout their careers due to 
participation in professional development courses. 

The most robust association was found between self-directed learning and the subdimension of multiple 
teaching. This suggests that these two subdimensions have a strong positive linear connection. This result is 
significant because it demonstrates that when multiple teachings are provided, self-directed learning increases 
as well. Self-directed learning refers to a large number of individual and group activities performed by students 
in the classroom and at home, without the direct involvement of the teacher (Kovalenko & Smirnova, 2015). To 
achieve this, teachers must have high multiple teaching competencies. 

According to the study’s findings, there are five unique categories of teachers based on their self-reported 
self-efficacy and teaching abilities. The first group is deemed moderately competent, suggesting they perceive 
themselves to be average in all categories relative to the other groups. The second group has low self-efficacy 
and mediocre teaching skills, except for innovative teaching and comprehensive teaching approach, where they 
score above average. The third group, termed ‘very competent, thinks themselves proficient in every subject 
area. The fourth group has low self-efficacy but outstanding teaching abilities, which is somewhat unexpected 
but demonstrates that low self-efficacy does not necessarily equate to bad teaching ability. The last group pos- 
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sesses high levels of self-efficacy but lacks unique teaching skills. While the findings are intriguing, the study has 
several limitations. The evaluation of self-efficacy and teaching skills, for instance, is reliant on self-reported data, 
which may be vulnerable to biases and mistakes. In addition, the study does not describe the criteria or measure- 
ments used to evaluate teaching skills, making it impossible to judge the validity and reliability of the findings. 
Moreover, the study did not examine the elements that lead to the variations in self-efficacy and teaching skills 
between the five groups, such as teacher training, experience, or motivation. Yet, the study's findings may be 
useful for informing teacher training and professional development initiatives. Teachers in the second category, 
for instance, may benefit from training that focuses on boosting their self-efficacy in areas other than innovative 
teaching and comprehensive teaching approach. To complement their already high levels of self-efficacy, the 
fifth group may benefit from training that stresses the creation of creative teaching strategies. In conclusion, the 
data reveal substantial variations in teacher self-efficacy and teaching skills, highlighting the need for treatments 
that address the unique strengths and limitations of individual teachers. In the literature review, no latent profile 
analysis study based on the competencies of Physics teachers was found. In the study conducted by Heuling et 
al. (2021), latent profile analysis was performed depending on the digital competencies of pre-service science 
teachers. According to the results of the study, 4 profiles were formed according to information & digital literacy, 
communication & collaboration, digital content creation, safety, and problem-solving competencies. Latent 
profile analysis was conducted by Tondeur et al. (2017) based on the level of technological pedagogical content 
knowledge of pre-service science teachers. According to the results of the analysis, pre-service teachers were 
gathered in two profiles as high and low proficiency. 


Conclusions and Implications 


The study explored educators’ self-perceptions, comprehensive teaching approach, and innovative teach- 
ing skills in physics. The findings revealed that teachers view themselves as very competent in both teaching 
self-efficacy and the emotional domain. They also consider themselves to be acceptable in the areas of multiple 
teaching, self-directed learning, and student-centeredness. In terms of comprehensive teaching approach, teachers 
feel that their planning and implementation skills are acceptable. However, there were no statistically significant 
correlations between Teaching self-efficacy and Multiple teaching and Teaching self-efficacy and Self-directed 
learning, as well as between Implementations and Multiple teaching and Implementations and Self-directed 
learning. Additionally, the study categorized teachers into five groups based on their self-perceptions and teach- 
ing skills. The groups ranged from moderately competent to extremely competent. The findings indicated that 
some teachers had low self-efficacy but excellent teaching skills, while others had high levels of self-efficacy but 
lacked innovative teaching abilities. These results provide valuable insights into teachers’ self-perceptions, teach- 
ing efficacy, and innovative teaching skills in physics, which can inform the development of effective professional 
development programs for teachers. 

According to the findings of the study, latent profile analysis provided teachers with relevant classification 
results. Future researchers could implement latent profile analysis using a variety of measuring instruments. In 
contrast, self-report data collecting instruments were utilized in this investigation. Classroom observations can 
be leveraged to develop measuring instruments for assessing teachers’ teaching skills. 
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Self-efficacy 

1. [could induce students’ interests in physics learning. 

2. | have sufficient professional physics teaching knowledge and competence to cope with students’ learning 
problems. 

3. My earnest teaching is the most important factor in promoting students’ learning outcome. 

4. |could create a harmonious and happy learning climate in physics education. 

5. lam capable of using different teaching methods for students in different levels. 

6. | know how to rapidly control class order. 

7. lamcapable and confident of improving students’ learning attitude towards physics education. 

8. |am-confident of improving students’ emphasis on physics education. 

9. Physics teachers present better influence on guiding physics learning 

10. | feel that |am capable of changing students’ incorrect physics knowledge. 

11. The effect of family environment on students could be overcome through education. 

12. |can develop the teaching effectiveness under existing environment and equipment. 


Innovative physics teaching 


OO" ON OT BG NOs 


| would often provide novel and stimulating materials. 

| would allow students freely select assignment topics related to the course. 
| often encourage students preceding self-evaluation. 

| would often arrange self-directed learning activities. 

| would often arrange diverse learning activities beyond physics teaching. 

| would utilize actual contingency for opportunity education. 

| would often teach with similes or analogy. 

| would provide diverse solutions to respond to students’ questions. 

| would ask questions to guide students’ multiple thinking. 


. [would normally accept students’ points of view. 
. Loften encourage students to accept others’ opinions. 
. | would particularly cultivate students’ responsibility, humor, and openness of mind when planning teaching 


activity. 


. [would often guide students to think with images. 

. [would normally introduce diversified teaching facilities. 

. [would normally allow students to apply various methods or media. 

. [would often plan different physics topics to train students’ ability development. 
. Loften encourage students through discussion and interaction. 

. | would normally plan diverse lessons and teaching activities. 


Comprehensive Teaching Approach 


SO NAW RWN > 
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Understanding students’ physics knowledge before lesson. 

Investigating students’ security before lab instruction. 

Making teaching records before instruction to cope with special situation. 

Practicing physics teaching according to teaching plan and schedule in the instruction. 

Providing definite action demonstration and explanation in the lab instruction. 

Systematically presenting materials from simplicity to complication in the instruction. 

Effectively distributing teaching time in the instruction to maintain fluent teaching-learning process. 
Providing adequate practice time in the instruction for students familiarizing material content. 

| would make different evaluation standards according to students’ individual differences. 

| would precede evaluation with diverse methods. 


. | would inquire whether students really understand teaching content from simplicity to complication. 
. | would adjust the teaching schedule or method according to students’ evaluation results. 

. [would precede self-evaluation according to teaching content. 

. [would express goodwill and assistance to students with learning difficulty or special needs. 


ASS https://doi.org/10.33225/jbse/23.22.668 


Journal of Baltic Science Education, Vol. 22, No. 4, 2023 


ISSN 1648-3898 /Print/ PHYSICS TEACHERS’ SELF-EFFICACY, INNOVATIVE TEACHING AND COMPREHENSIVE 
TEACHING APPROACH: LATENT PROFILE ANALYSIS 
ISSN 2538-7138 /oniines (PP.668-681) 


15. | would appreciate and encourage students with the learning progress. 

16. Providing students with opportunities for peer cooperation and interaction in the instruction. 

17. | would provide students with opportunities for expression and decision making with a democratic education 
style. 

18. | would avoid sarcastic and negative words to criticize students. 

19. Establish teaching rules and procedures before instruction and ask for students’ compliance. 

20. Maintaining good class order in the instruction. 
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Abstract. /nitial teacher education should 
mould well-rounded teachers proficient 

in scientific investigations, and advocate 
sustainability amid global ecosystems’ 
degradation. The research sought to 
explore pre-service science teachers’ 
views concerning scientific investigations 
and sustainable development goals, and 
synergy between skills and different aspects 
of scientific inquiry within the context 

of the mangroves ecosystem studied. A 
four-part questionnaire was administered 
to 82 students registered for a Bachelor 

of Education degree. It included items 

on teachers’ confidence in planning and 
conducting scientific investigations, types 
of scientific investigations, sustainable 
development goals, science process skills 
and the different aspects of scientific 
inquiry. Statistical analysis of the data 
showed the importance of a constructivist 
approach, learner engagement and 
discerning and controlling variables 

when investigating scientific phenomena. 
However, only a fair number of teachers 
could correctly provide the scientific 
investigation they conducted during 

the mangroves study, the sustainable 
development goals embedded in it, and 
the synergy between science process skills 
and the aspects of scientific inquiry. The 
findings have implications for teacher 
education in terms of potential challenges 
in teachers’ understanding of scientific 
investigations, the synergy between them 
and sustainability, and science process 
skills that meld with the aspects of scientific 
inquiry. 
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Introduction 


Initial teacher education should prioritize key competencies in teachers 
if the intention is to mould an all-round science teacher. Such competencies 
should be focused on knowledge and understanding, skills, values and prin- 
ciples, methods and procedures relevant to a discipline (Molefe & Stears, 2014). 
They (competencies) should be emphasized in Environmental Education (EE) 
and education for sustainable development (ESD), where teacher professional 
development praxis and research have assumed central importance. First, 
there is a much-needed synergy between EE and science education due to 
concerns about sustainability (Wals et al., 2014) and the need for education 
about, in and for the environment. Indeed, EE has long endorsed a multidis- 
ciplinary view of the environment and most importantly, the acquisition of 
skills and solving of environmental problems (UNESCO, 1980). The typology 
thereof is analogous to science education’s emphasis on skills development 
through scientific investigations (Scil) and connections to science and society 
(Pedretti, 2014). Second, skills together with methods implemented during 
EE activities might be applicable to science processes (McDonald & Domin- 
guez, 2010). Science processes had long been viewed as an amalgamation 
of skills and procedures to be developed and utilized in Scils (So, 2003). For 
Colley (2006), science content and process within the context of ecology 
might entail, for instance, collaborative fieldwork and Scils. Such Scils may 
be of great importance when addressing indigenous forests as focal points 
for aligning sustainable development goals (SDGs) with carbon mitigation 
and conservation (Cf. Smithwick, 2019). 

The synergy of SPS and Scil itself is not new (see Arena, 1996). SPS 
might not only be key in cognitive development (e.g., inquiry, reasoning, 
and problem-solving skills) (Ozgelen, 2012) and gathering new knowledge 
but also in the understanding of scientific processes and methods (Cipkova 
& Karolcik, 2018) during fieldwork. For teachers to be able to grasp and then 
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address environmental problems, they should first have an in-depth knowledge and understanding of EE and a 
wide range of skills (e.g., thinking skills and SPS) (Dresner et al., 2014). Unfortunately, a conceptual understanding 
of SPS can be a challenge to teachers (Cipkova & Karolcik, 2018; Hafizan et al., 2012; Mumba et al., 2018) possibly 
because of lack of confidence and limited understanding of the skills (Ambross et al., 2014). On the other hand, the 
challenges concerning the implementation of Scils have primarily been rooted in teachers’ mindset stuck in a rigid 
pedagogy (Ramnarain, 2011) and views about the scientific method and its role in the rigour of these activities 
(Scils) (Gobalek & Amrane-Cooper, 2011), and lack of confidence in planning and implementing them for learner 
learning (Pepper, 2013). EE itself has also presented a challenge to teachers (Reddy, 2011). This conglomerate of 
challenges thereof points to the need for teachers’ acquisition, for instance, of relevant skills and understanding 
of scientific concepts. Most importantly, they suggest a need to disrupt the pre-service teachers’ understanding of 
Scils and the embedded SPS during ecosystem-based studies, as well as the supremacy of science when working 
on environmental problems. 

The new era of SDGs put emphasis on humanity rethinking its place in the world, hence the increase in the 
mobilisation of green life skills today (Kwauk & Casey, 2021). This is overdue considering that climate change has be- 
come the fundament global issue that threatens countries’ sustainable development, hence a need to problematise 
teachers’ roles and their understanding of the content of climate change education (Apollo & Mbah, 2021). In South 
Africa, mobilisations on methods and processes to lobby for change-oriented learning towards better environmental 
sustainability practices and/or environmental learning in a wide range of contexts (see Fundisa for Change Programme 
[FCP], 2013; Rosenberg et al., 2008; Shava & Schudel, 2013; Vogel et al., 2013) have since stimulated the development 
of science process skills and implementation of Scils. The Natural Sciences curriculum itself advocates responsibility 
towards the environment while learners develop scientific skills and processes associated with investigating natural 
phenomena (Department of Basic Education [DBE], 201 1b). Thus, it is reasonable that the curriculum envisages compe- 
tent teachers whose praxis is enshrined in the understanding of “the need for using scientific knowledge responsibly 
in the interest of ourselves, of society and the environment” (DBE, 2011b, p. 9). In other words, the teachers should 
show both interest in science and environmental awareness, and proficiency in scientific knowledge (FCP, 2013). They 
should also understand that fieldwork is a signature pedagogy for future outdoor EE teachers (Thomas & Munge, 
2017). Most importantly, they should understand that the complexity of the environment in a broader sense requires 
an equally extensive view of Scils and the associated development of both SPS and SDGs. 


Literature Review 


Although EE has along history, it could be envisaged as a newcomer, particularly in science education. It had 
to navigate, for instance, the hegemony of school-based disciplines before it could forge a mutual and compatible 
agenda with science education (Pedretti, 2014). The Tbilisi Conference remains the starting point for the launch of 
its (EE) programme in which knowledge, skills, attitudes, environmental problems and, for instance, Scils became 
inextricably intertwined in a quest for environmental awareness (UNESCO, 1980). Thus, it is reasonable that the 
emphasis in EE is still on methods that “develop and support environmentally related attitudes, values, awareness, 
knowledge, and skills that prepare people to take informed action on behalf of the environment” (Ardoin et al., 2020, 
p. 1; emphases added). The methods such as ecological investigations have long played a major role in develop- 
ing learners’ thinking abilities (Kostova & Atasoy, 2008). In elaboration, it can be argued that its (EE) premises are 
enshrined in environmental awareness (see Littledyke, 2008; Moyo & Masuku, 2018; Soto-Cruz et al., 2014; Thor & 
Karlsudd, 2020). Inquiry-based learning might dominate ecological investigations (Kostova & Atasoy, 2008). Thus, 
it could be argued further that EE might form a springboard for the synergy between inquiry-based learning and 
ecology (Taylor & Bennett, 2016) and the scientific process (Tang et al., 2009; Thomas, 2012) when developing learn- 
ers’ ecology-based content and SPS (Colley, 2006). EE might also shed a light on the synergy between mangroves 
and climate change (Wimmler et al., 2021), SDGs and this (mangroves) ecosystem (see Dandabathula et al., 2021) 
and the Scils themselves. 

Outdoor activities had long been endorsed as means of enabling learners to use skills to promote a holistic 
understanding of the environment through hands-on experiences (Meredith et al., 2000). Fieldwork, in particular, 
could enable interaction between affective, psychomotor and cognitive domains in which learners are afforded 
meaningful learning of biodiversity (Scott et al., 2011), particularly in relatively less studied marine and freshwater 
ecosystems (Millenium Ecosystem Assessment, 2005). Indeed, Rog et al. (2017) pointed to the need to recognise 
mangroves’ cross-realm importance amid a considerable knowledge gap in their ecology. After all, mangroves 
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remain refuge habitats than mere marine ecosystems. Most importantly, mangroves may play a major role in green- 
house gases sequestration and alleviation of natural disasters (see Dandabathula et al., 2021) - a reason they had 
been advanced as part of the fundamental strategies concerning climate change such as REDD+ and blue carbon 
(Alongi, 2012), hence their connection with SDG 13 (climate action). That said, there are other appropriate teaching 
and learning methods that have been proposed to study ecological issues in tandem with fieldwork. Yli-Panula et 
al. (2019) pointed to the importance of inquiry-based learning, argumentation, group work, active participation 
and thinking skills as essential. Their argument is reasonable. Inquiry literacy and that of the environment are key 
for teachers to be able to develop their learners’ understanding of the environment in a broader sense (Kidman & 
Casinader, 2019). The implication is that teachers’ study of mangroves ecosystem (through Scils) may enable them 
to be actively involved in learning scientific ideas and concepts (Abrahams & Millar, 2008) and promote critical 
thinking skills as they reflect on, for instance, the fauna of the ecosystem (Puche & Holt, 2012). They may not only 
enable collaborative learning but also inquiry and fieldwork. Indeed, the popularity of fieldwork gained traction 
in science education due to its ability to accommodate inquiry-based learning in which learners may engage in 
Scils (Remmen & Frayland, 2014), hence developing SPS. 

Teachers should have competency in subject matter and scientific concepts as well as primary skills such 
as SPS (Gultepe, 2016). In terms of SPS, an average South African is expected to implement them (SPS) in the 
post-apartheid education system (Ambross et al., 2014). This is reasonable considering that SPS have remained 
the fundamental part of research over the last seven years (Cipkova & Karoléik, 2018; Gultepe, 2016; Mulyeni et 
al., 2018; Ongowo, 2017; Susanti et al., 2018; Tilakaratne & Ekanayake, 2017; Shahali et al., 2015; Wahyuni et al., 
2017) despite the long-established debates on processes and content in science education (see Millar & Driver, 
1987; Wellington, 1989; So, 2003). Science process skills are means of investigating phenomena and constructing 
science concepts (Gultepe, 2016). They are normally grouped as either basic or integrated skills. For Shahali et al. 
(2015), the key to the successful utilisation of SPS and implementation of inquiry-based teaching by teachers lies 
in their (teachers’) conceptual and operational understanding of the skills (SPS) themselves. Recently, Molefe and 
Aubin (2021a) highlighted the importance of conceptual understanding and context when SPS are taught and/ 
or developed. It makes sense, therefore, that content and process should be incorporated into the teaching of sci- 
ence, as both are equally important (Gultepe, 2016) in any given context. Indeed, SPS can be developed in the EE 
context in which teachers seek to make sense of and solve environmental problems today. Their intersection with 
the 21* Century skills makes them (SPS) more invaluable than ever. As we venture into the digital age marked by 
technology and knowledge explosion, SPS and scientific literacy might provide a springboard for the development 
of 21 Century skills (Turiman et al., 2012). For instance, the development of SPS through an investigative study 
of mangroves may form the basis for the cultivation of collaborative work, critical thinking, and communication 
skills. Thus, it could be argued that the development of SPS by teachers should be linked to EE and engaged in 
and developed subsequent to conceptual understanding. In this case, that could be a holistic understanding of 
mangroves ecosystems in a broader sense using suitable teaching and learning strategy. 


Conceptual Frameworks 


The present research entailed a conglomerate of concepts - Scils, SDGs and SPS - which were linked to en- 
able the researchers to come up with a better explanation of their (concepts) relationship within the context of a 
phenomenon studied - mangroves ecosystem. The concepts formed the basis for frameworks that were deemed 
to be suitable for this research. That said, the researchers acknowledge that literature has different ways of listing 
and framing SPS (e.g., DBE, 2011b; Gultepe, 2016; Molefe & Aubin, 2021a, 2021b; Saban et al., 2019) and Scils (Frost, 
2010; J. Lederman et al., 2019; Moeed, 2013; Pepper, 2013; So, 2003; Watson et al., 1999). 


Scientific Investigations, Sustainable Development Goals and Science Process Skills 


As we migrate into the 4" Industrial Revolution (4IR), there is a need for rethinking of the curriculum in higher 
education in terms of creating students who can collaborate in diverse teams, have creative minds, and navigate 
through rapidly changing information and global challenges (Penprase, 2018). Thus, teachers’ proficiency in SPS, 
understanding of aspects of scientific inquiry and confidence in planning and conducting Scils cannot be overempha- 
sised. Amid the rhetoric debates on the pedagogical impact of 4IR in higher education, the reality (see J. Lederman 
etal., 2019) is that teachers might not hold informed views of the scientific inquiry and Scils. For instance, teachers 
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might understand science investigation or the scientific method as a stepwise and sequential process (Moeed, 
2013; Molefe & Aubin, 2021a, 2021b). Molefe and Aubin (2021a) went further to argue that teachers might also 
encounter problems in understanding Scils in terms of SPS embedded in the scientific process steps of questioning, 
design of experiments and making conclusions and inferences. On the other hand, UNESCO (2021) puts sustain- 
able development at the heart of humanity’s change of course as it reimagines its future in this planet. UNESCO 
(2020) also provides ESD for 2030 roadmap, which succinctly maps out the contribution of (science) education to 
the realisation and enactment of the SDGs. The researchers propose that the aforementioned issues related to the 
key concepts of this research need to be interrogated if the intention in science education, as we venture into the 
21% Century, is to mould an all-round teacher that also advocates for ESD. 

Conceptually, Scils undergirded the present research. As referred to earlier, our latest work showed that the 
concept of Scil has not been fully understood by science teachers. Thus, the researchers drew from Pepper's (2013) 
and Roberts's (2004) works for a framework on teachers’ confidence in planning and conducting science investiga- 
tions. It should be noted that the two skills thereof are part of the SPS stipulated in the South African science cur- 
riculum (Table 1; DBE, 2011b). Furthermore, the researchers drew from Baez et al. (1987), Frost (2010), and Watson 
et al. (1999) for a framework concerning the types of Scils (e.g., Classifying and identifying, Exploring, Fair testing and 
comparing, Making things or developing systems and Pattern seeking) the teachers thought they had conducted 
during their mangroves study. 


Table 1 
14 Scientific Skills Stipulated in the South African Science Curriculum 


Science process skills 


Accessing & recalling information 
Observing 

Comparing 

Measuring 

Sorting & classifying 
Identifying problems & issues 
Raising questions 

Predicting 

Hypothesizing 

Planning investigations 
Doing investigations 
Recording information 
Interpreting information 


Communicating 


As we rethink our roles in the collapse of ecosystems and climate change, a broader perspective of Scils 
was essential in terms of SDGs. Thus, the study was further framed using ideas by UNESCO (2020) for SDGs. SDGs 
have become a holy grail for sustainable development (SD) and heralded great hope for humanity against global 
environmental issues. They were investigated within the context of Scils related to mangroves ecosystem. Indeed, 
Paristiowati et al. (2022) eloquently show the synergy between the SD theme, sustainability perspectives, field- 
work, scientific skills (e.g., SPS), 21** Century skills, and environmental awareness within a course programme. The 
researchers went further to draw from DBE (2011b) and J. Lederman et al. (2019) for the teachers’ understanding 
of the aspects of the scientific inquiry through the lens of embedded SPS. 


Research Focus 


The present research was based on pre-service teachers’ fieldwork-based experiences, Scils and SDGs, and 
SPS and different aspects of scientific inquiry within the context of a mangroves-based ecosystem. The researcher 


685 


https://doi.org/10.33225/jbse/23.22.682 LT 


Journal of Baltic Science Education, Vol. 22, No. 4, 2023 


A NEXUS OF SCIENTIFIC INVESTIGATIONS, SCIENCE PROCESS SKILLS, AND SUSTAINABLE ISSN 1648-3898 /Print/ 
DEVELOPMENT GOALS: PRE-SERVICE TEACHERS’ VIEWS CONCERNING MANGROVES 
FIELDWORK ISSN 2538-7138 /Online/ 


(PP.682-700) 


has chosen to focus on mangroves because anthropogenic impacts and natural changes are of great concern with 
regard to them in South Africa and globally (Hoppe-Speer et al., 2015) in the 21% Century. The concern has since 
pointed to the need to adhere to the longstanding call for the integration of ESD in teacher education (UNESCO, 
2017). UNESCO (2017) puts emphasis on ESD’s content and its teaching and learning methods. In science education, 
the methods may entail the utilisation of Scils and SPS while studying ecosystems. Indeed, the study of mangroves 
ecosystem enabled the present teachers to utilise several SPS to study its flora and fauna. The teachers’ views were 
also explored because, for decades, they (teachers in general) have been part of the empirical work on biology 
fieldwork (Lock, 2010; Tilling, 2018). In this case, they studied different ecosystems that included mangroves dur- 
ing their annual three-day fieldwork. This component (fieldwork) is part of an ecological studies course offered at 
an institution in KwaZulu-Natal. 

It should be noted that global problems and threats negatively impact climate and the environment (Ivanov, 
2018). Thus, it is reasonable that today there is a proposal for the inclusion of not only scientific literacy but interest 
in science and inquiry-based teaching in programmes intended to increase learners’ environmental awareness (List 
et al., 2020). Such programmes may include Scils on the life essential for the sustenance of mangroves ecosystems 
(Rog et al., 2017). They may also review factors that had shaped mangroves’ decline, particularly in South Africa 
(Osorio et al., 2016). After all, mangroves ecosystems”blue carbon storage and sequestration capability are important 
regulatory services since 2011 because of global climate change mitigation” (Sharma, 2018, para. 2; emphases added). 


Research Aim and Research Questions 


The aim of this research was to draw from pre-service teachers’ fieldwork-based experiences to establish their 
views in relation to Scils and SDGs, and the synergy between SPS and different aspects of scientific inquiry within 
the context of mangroves-based ecosystem. 

The research sought to answer the following research questions in relation to an ecological studies course: 


1) What did pre-service teachers perceive as their level of confidence in planning and conducting a sci- 
entific investigation for learners learning? 

2) What type of scientific investigation was conducted by the pre-service teachers during their mangroves 
ecosystem study? 

3) What sustainable development goals and science process skills did the pre-service teachers identify 
as embedded in the scientific investigation they contacted? 


Research Methodology 
General Background 


This research adopted an approach that was quantitative in nature. This approach can accommodate quan- 
titative questionnaires that can be used to conduct surveys with teachers (Mertens, 2014). For Cohen et al. (2018, 
p. 471), “the questionnaire is a widely used and useful instrument for collecting survey information, providing 
structured, often numerical data...” It is therefore reasonable that survey research can provide numeric descrip- 
tions of patterns or opinions from a sample of a particular population under study (Creswell, 2014). The present 
survey was adopted as a research strategy and utilised the questionnaire as an instrument to gather data on the 
pre-service teachers’ views about Scils and SDGs, and SPS embedded in aspects of scientific inquiry within the 
context of mangroves ecosystem studied. Although surveys usually yield information that is essentially statistical 
in nature (Neuman, 2014), the present questionnaire had open-ended questions. The questions allowed the teach- 
ers to provide reasons for their views. A questionnaire was utilised in this research due to its appropriateness in 
research based on SPS (Fugarasti et al., 2019) and teachers’ views about Scils (Pepper, 2013). Furthermore, it could 
also be used to study the understanding of scientific inquiry among learners (J. Lederman et al., 2019; Penn et al., 
2021) and students (J. Lederman et al., 2021; Penn & Ramnarain, 2022), and for incorporating ESD’s theme in pre- 
service teacher science programme (Paristiowati et al., 2022). 
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Participants 


In this research, data were collected from 82 conveniently selected pre-service teachers who enrolled for 
an ecological studies course at an institution in KwaZulu-Natal, South Africa. Careful selection of respondents rather 
than sample size is key amid several factors that influence sample size decisions (Memon et al., 2020). The sample 
size enabled us to compute the statistical tests we wished to make. However, the sample selection might have 
weakened external validity because the present teachers completed the questionnaire as part of their fieldwork 
exercise. The research was primarily on the participants’ views about Scils, not their (participants’) characteristics 
- a reason demographic information was omitted from the research’s questions and design. It should be noted 
that iterative practice and scaffolding approach, particularly in relation to SPS (Coil et al., 2010), Scils and SDGs are 
key. The present teachers had studied all the aspects thereof in two content and method courses offered at the 
institution. Thus, the teachers were suitable participants, and they were assured of absolute anonymity. 


Instrument and Procedures 


The present research utilised a four-part questionnaire, which was tailored to meet the content of interest 
in the survey - Scils, aspects of scientific inquiry, SPS and SDGs. It was important that the questionnaire provided 
complementary and/or supplementary information to the numerical data collected. Thus, the questionnaire was 
designed to allow the teachers to provide reasons for their responses in two of its four parts. Apart from SDGs, 
it also had detailed descriptions of SPS stipulated in the South African science curriculum (DBE, 2011b; also see 
Table 1) and various types of Scils (Frost, 2010; Watson et al., 1999). Part one, a 5-point Likert scale for agreement 
or otherwise that was adopted from Pepper (2013), and Part two provided quantitative data concerning Scils. Parts 
three and four provided both quantitative and qualitative data on the nexus between Scils and SDGs, and aspects 
of scientific inquiry and SPS, respectively. The purpose of the questionnaire was to capture the teachers’ perceived 
confidence and ability to plan Scils for their learners, and the Scil they thought they used when studying mangroves 
ecosystem. Furthermore, it was utilised to elicit the primary SDGs that the teachers thought were associated with 
the mangroves-based investigations and two fundamental SPS that meld with each aspect of scientific inquiry. The 
students were requested to provide reasons for their responses concerning SDGs embedded in their investigation 
on mangroves and rationalise why the selected SPS fit into a given scientific inquiry’s aspect. 

Mertens (2015) pointed to the importance of clarity and pilot testing the questionnaire in surveys. Assessment 
by a panel of experts can also be used when establishing an instrument's face and content validity (Creswell, 2008). 
In this research, the questionnaire was piloted with eight pre-service teachers that enrolled for the same course 
during the preceding year. The subsequent findings were further interrogated at a conference (see Molefe, 2022). 
The post-conference questionnaire was further piloted with four students who demonstrated for the course. It 
should also be noted that three academics that teach Biology, Physics and Chemistry courses were invited to as- 
sess the researchers’ views concerning expected SPS and the rationale behind their selection for Part four of the 
questionnaire. The refined questionnaire was administered to the teachers after multi-ecosystem fieldwork. Ethical 
clearance for the research was used. 


Data Analysis 


All data were analysed using R. First, R was used because the data obtained required an extensible statisti- 
cal package that could enable the researchers to compute descriptive and analytical functions, which they needed 
to answer the questions of the research. Second, it was used because of its wide range of use in statistical tech- 
niques. Chi-squared tests can be conducted to “see if frequencies observed are statistically different or by chance 
alone” (Cohen et al., 2018, p. 789). In this research, it was important that the researchers verified that the answers 
were not given randomly for various questions. Thus, the preliminary results for Scils, SPS and SDGs were based 
on Chi-squared tests on the frequencies. Furthermore, binomial tests on the maximum frequencies observed for 
the aspects of scientific inquiry were also computed. The tests were corrected using a Bonferroni correction. 
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Chi-squared Test for Uniformity for Scils and SDGs 
Variables ad df p-value 

CP1: Using a constructivist model to plan science (e.g., Natural or Life Sciences) units of work 50.387 4 .00001 

CP2: Planning to engage learner interest in investigating in science 50.234 4 .00001 

CP3: Developing an investigation in science 35.581 4 .00001 

CP4: Identifying independent, dependent and control variables in a science investigation 46.202 4 .00001 

CP5: Recognising a fair test in a science investigation 41.299 4 .00001 

CP6: Developing a fair test in a science investigation 36.078 4 .00001 

CD1: Teaching science investigation concepts 74.707 4 .00001 

CD2: Facilitating learner discussion and interpretations of science observations 85.439 4 .00001 

CD3: Providing opportunities to extend learners’ understanding of science observations 66.537 4 .00001 

CD4: Assessing learner learning in science investigations 69.829 4 .00001 

CD65: Identifying learner strengths and weaknesses in science investigations 74.829 4 .00001 

CD6: Scaffolding learner learning in science investigations 72.756 4 .00001 

TSI: Types of Scientific Investigation 59.603 4 .00001 

SDGs: Sustainable Development Goals 72.519 16 .00001 


Research Results 


It was important that a test was made to check whether the answers concerning the teachers’ levels of 
confidence in planning (CP) and conducting (CD) Scils, the type of Scil (TSI) done concerning mangroves and the 
associated SDGs were given randomly. Table 2 shows the results of the Chi-squared test. The respective p-values 
(p < .00001) showed that the answers for those variables were not given randomly. 


Pre-service Teachers Perceived Levels of Confidence in Planning and 
Conducting a Scientific Investigation for Learners Learning 


In addition to the uniformity test, the researchers further explored the dataset. Table 3 shows that, in general, 
the present teachers’ perceived levels of confidence in planning and conducting a Scil for their learners were at 
least moderate. That said, it was interesting that most of the 82 teachers rated themselves as confident concern- 
ing planning a Scil in terms of Using a constructivist model to plan science (e.g., Natural or Life Sciences) units of work 
(84%), Planning to engage learner interest in investigating in science (89%) and Identifying independent, dependent 
and control variables in a science investigation (85%). Even more interesting was the teachers’ perception of the 
level of their confidence in conducting a Scil. They rated themselves as confident and the associated variables (i.e., 
CD1-CD6) were virtually uniform (around 83%). 
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Table 3 
Frequency Table for CP and CD 


Variables Frequency 
No confidence Limited confidence Moderate Confident Very confident 
confidence 
CP1 0 0 13 50 19 
CP2 0 0 9 36 37 
CP3 0 3 16 36 27 
CP4 0 1 11 25 45 
CP5 1 1 11 37 32 
CP6 0 3 19 40 20 
CD1 1 1 10 38 32 
CD2 0 1 13 36 32 
CD3 0 1 15 37 29 
CD4 0 1 13 36 32 
CD5 0 1 11 35 35 
CD6 0 1 12 37 32 


Type of a Scientific Investigation Conducted by the Pre-service Teachers during Their Mangroves Ecosystem Study 


The study of several ecosystems, which included mangroves, built on practical and theoretical aspects con- 
cerning problem-based learning, scientific inquiry and Scils was done at the institution. The present teachers also 
studied issues around biodiversity, global climate change, various ecosystems, and SDGs. Thus, one would have 
expected their reflections on Scils to be effortless. Table 4 shows that most of the teachers (57%) gave Classifying 
and identifying most often than any other Scil. Exploring was the second most (38%) often given type of Scil. 


Table 4 
Frequency Table for Scil Associated with Mangroves Ecosystem Studied by the Teachers 


Types of scientific investigations Frequency 
Classifying and identifying 47 
Pattern seeking 0 
Exploring 31 
Fair testing and comparing 4 
Making things or developing systems 0 


Sustainable Development Goals and Science Process Skills Pre-service Teachers Identified as Embedded in the 
Scientific Investigation they Contacted 


In this study, the SDGs of interest were those related to Climate action, Life below water and Life on land (i.e., 
SDGs13, 14 and 15; the latter was given as an example in the questionnaire) due to their indubitable connection 
with the mangroves ecosystem studied. The intention was to establish the teachers’ views concerning SDGs (and 
ESD) in tandem with a particular ecosystem-based Scil. The results showed that SDG13 (27%) and SDG14 (40%) 
were given most often (Table 5). Two main themes - Mangroves (ecosystem) and other life forms and the Impact of 
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climate (change) on the ecosystem and vice versa - were elicited from the most given reasons for the teachers’ choices 
of the two leading SDGs. Below are some of the written descriptions associated with the themes: 


Table 5 


Table for SDGs Associated with a Mangroves Ecosystem Studied by the Teachers 


SDG14: 


SDG14: 
SDG14: 


SDG14: 
SDG13: 


SDG13: 


Variable Number of responses for each SDG 
SDG1: No poverty 1 
SDG2: Zero hunger 0 
SDG3: Good health and well-being 3 
SDG4: Quality education 3 
SDG5: Gender equality 0 
SDG6: Clean water and sanitation 7 
SDG7: Affordable and clean energy 3 
SDG8: Decent work and economic growth 5 
SDG9: Industry, innovation and infrastructure 0 
SDG10: Reduced inequalities 0 
SDG11: Sustainable cities and communities 4 
SDG12: Responsible consumption and production 0 
SDG13: Climate action 22 
SDG14: Life below water 33 
$DG15: Life on land 0 
SDG16: Peace, justice and strong institutions 0 
SDG17: Partnerships for the goals 1 


Mangroves are adapted to live in coastal regions, [sic] they are able to adapt in wet and salty ecosystems hence, 
they are able to maintain life below water (Teacher 1). 

...we find mudskippers on mangrovel[s] swamps living in and out of the water (Teacher 2). 

.. organisms like crustaceans are attached to the mangrove forests because of the variety of food, cooler water 
and safety that is found among the roots (Teacher 3). 

Mangroves require a certain climate for them to survive (Teacher 4). 

.../f they [sic] is climate change due to global warming or any other cause they will not be present hence why 
| choose climate action as to help reduce the causes of climate change due to human activities in order to 
prolong and preserve ,[mJangrove ecosystems [sic] (Teacher 5). 

Interms of climate management, mangrove forests play a critical role since they serve as carbon-rich buffer zones 
between land and oceanas in this waterlogged environment leaves and other detritus do not decompose easily 
by doing so, the carbon in this organic matter is prevented from escaping (Teacher 6). 


It was important that further tests were made to check whether the answers concerning the teachers’ views on 
SPS embedded in the different aspects of inquiry were given randomly. Table 6 shows the results of binomial tests. 
The respective p-values (p < .05) showed that the answers for those aspects were not given randomly. The research- 
ers further explored the dataset, now for the frequencies of teachers’ answers and the reasoning behind them. 

Appendix A shows that, in general, the teachers were not able to correctly select the SPS that are embedded 
in various aspects of scientific inquiry. It was only in the case of Observing (55%) and Scientific investigations all 
begin with a question but do not necessarily test a hypothesis (SIQ), Interpreting information (48%) and Research 
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conclusions must be consistent with the data collected (RCD), and Accessing & recalling information (49%) and Ex- 
planations are developed from a combination of collected data and what is already known (EDK) that about half 
of the teachers seemed to have been able to select the expected SPS. 


Table 6 
p-values of Binomial Tests on the Maximum Frequencies Observed for the Aspects of Scientific Inquiry 


Aspects of scientific inquiry Maximum frequency p-value 
observed 
SIQ: Scientific investigations all begin with a question but do not necessarily test a hypothesis 12 .001 
SIP: Inquiry procedures are guided by the question asked 5 024 
SIR: All scientists performing the same procedures may not get the same results 7 .001 
IPR: Inquiry procedures can influence the results 5 024 
RCD: Research conclusions must be consistent with the data collected 14 001 
SDE: Scientific data are not the same as scientific evidence 14 001 
EDK: Explanations are developed from a combination of collected data and what is already known 6 .003 


Discussion 


When teaching in and across disciplines, it is important that students are engaged in purposeful Scils in au- 
thentic contexts to nurture collaborative and experiential learning in which scientific inquiry forms the basis for 
their views and understanding of the world (Crawford et al., 2005). That said, MacDonald and Dominiquez (2010) 
had long pointed to the importance of EE and its methods in teacher preparation programmes. While EE field trips 
are hailed as a centrepiece and key for science methods and the development of a conglomerate of skills, they 
address the fundamental goals of EE (Rebar & Enochs, 2010). Most importantly, ESD, which broadens the concept 
of EE, has assumed a mantle for transformative education that underpins ideals and principles that advocate the 
sustainability mindset (e.g., environmental protection, ecological sustainability, climate change, etc.) (SADC, n.d.). 
That explains why reorienting our focus on EE and SD has been and/or is the basis for debates on the environment 
at national (Mabhaudhi et al., 2019; Mpandeli et al., 2018; SADC, n.d.) and international levels (Boca & Saracli, 2019; 
UNESCO, 2020, 2021). The researchers acknowledge that the concept of EE and teacher education (Bodzin et al., 
2010; Reddy, 2011), SDGs (UNESCO, 2020), the environment and the associated development of skills and knowledge 
(Colley, 2006; VanLeuvan & McDowell, 2000), the development of SPS in teacher education (Molefe et al., 2016) and 
inquiry (Artayasa et al., 2017), the Scils (Baez et al., 1987; Frost, 2010; Watson et al., 1999) and teachers’ views of them 
(Moeed, 2010), and phases (Pedaste et al., 2015) and/or aspects of scientific inquiry (see J. Lederman et al., 2021; 
Penn & Ramnarain, 2022) are hardly new. However, teachers’ views of the mangroves ecosystem in which Scils and a 
nexus of them (Scils) and SDGs, and SPS and aspects of scientific inquiry are investigated, remains uncharted terrain. 
This research sets a precedent of the aspects thereof for another research to learn from. 

The importance of investigative work in school science, which can be traced to the Nuffield projects of the 
20" century, has long been part of the policies of some global north countries (Toplis & Cleaves, 2006). It became 
part of the core practical work tasks that include fieldwork (Science Community Representing Education [SCORE], 
2008). The South African school science curriculum’s emphasis on practical tasks and investigations has since echoed 
such countries’ reform trends in science education. In Life Sciences, learners are expected to be “able to plan and 
carry out investigations as well as solve problems that require some practical ability” (DBE, 2011a, p. 15; emphases 
added). The present study’s results showed teachers’ confidence in the two skills thereof. Considerable confidence 
was evident with regard to the items related to the constructivist model (CP1), planning to engage learner interest in 
investigating science (CP2) and variables (CP4). Similar findings concerning the teachers’ proclaimed confidence in 
the constructivist model were also found in Pepper (2013). Furthermore, teachers may recognise potential teach- 
ing and learning models to engage their future learners in Scil. Hence, the present teachers’ responses may also be 
understood in terms of emphasis on the constructivist approach to teaching and learning in our science method 
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courses. The influence of the South African Natural Sciences curriculum, whose scientific process is tailored on fair 
testing and comparing (Molefe & Aubin, 2021b; also see CP5 and CP6), resonates with variables (CP4). 

While the present teachers’ confidence concerning Scils is commendable, previous research by Toplis and 
Cleaves (2006) had shown that learners could be alienated from Scils even in developed countries. In the developing 
countries such as South Africa, emphasis on Scils is theoretical rather than practical, and that might be a norm due to 
several contextual factors that include avoidance of hands-on minds-on practical activities (J. Lederman et al., 2019). 
Furthermore, the proclaimed use of the constructivist model (Table 2) might contradict the challenges teachers often 
have concerning the conceptualisation and facilitation of Scils (Cf. Kibirige et al., 2022). Thus, it was important that 
the present teachers were put under a microscope concerning Scils within the context of the mangroves’ ecosystem. 

Most teachers viewed Classifying and identifying as the Scil they executed during the mangroves study (Table 
4). That is reasonable because they observed various species that adapted to the ecosystem. Most importantly, they 
identified and classified those species using their characteristic features suitable for the ecosystem. It was interesting 
that Exploring was the second most selected Scil considering that it involves making careful observations over time 
(Watson et al., 1999, 2006). The wrong selection justifies Molefe and Aubin's (2021b) argument that teachers might 
understand Exploring in terms of investigating species rather than making observations of them. J. Lederman et al. 
(2019) pointed to the challenge of understanding scientific inquiry itself for teachers. This is probably due to the 
cognitive load associated with conducting Scils hence a need for scaffolding practice and iterative practice (Jerrim 
et al., 2019). 

Today “negative trends in biodiversity and ecosystems [e.g., mangroves] are predicted to undermine 80% of 
the Sustainable Development Goals targets related to...climate, oceans and land” (Ometto et al., 2022, p. 2375). 
Thus, the present teachers were further challenged to now connect SDGs to their Scil. As referred to earlier, less 
than 50% of the teachers could show the connections between the studied mangroves and SDGs 13 and 14. They 
drew from the reciprocal connection between climate (change) and the ecosystem, and Mangroves (ecosystem) and 
other life forms to support their views. The connections are reasonable because the significant role of mangroves 
in the environment and the impact of climate change on them are well documented (Ahmed et al., 2022; Alongi, 
2012; Friess et al., 2022). That said, the low percentages thereof, and the spread responses (Table 5) suggest a 
need to interpret the results here with caution. Notwithstanding, the spread responses might actually highlight 
the argument that mangroves are connected to all the SDGs (see Dandabathula et al., 2021), and interestingly 
Covid-19 (McHarg et al., 2022). 

The connection between Scils, scientific inquiry and SPS cannot be overemphasised in teacher education (Cf. 
Dudu &Vhurumuku, 2012). Due to the challenging nature of the highlighted concepts thereof, it was important that 
the teachers supported their views (see Appendix A - Sample of reasons given) concerning SPS embedded in the 
aspects of inquiry. Appendix A shows that, in general, the teachers were not able to select more than one expected 
SPS for the aspects - Research conclusions must be consistent with the data collected (RCD) and Scientific data are 
not the same as scientific evidence (SDE). Similar to SDGs results, the teachers’ responses were spread. This explains 
the low percentages of the dominating unexpected SPS (i.e.,42% and 38%) selected for the two aspects thereof and 
the rest. 42% of the teachers paired Recording information with RCD. The combination points to the influence of the 
South African science curriculum concerning SPS and the scientific process:“Recording information [is] recording data 
from an investigation...’, and inferences about recorded observations form the basis for conclusions (DBE, 2011b, p. 
11-12; emphases added). For SDE, Gyllenpalm et al. (2021) discussed the ambiguity of the terms - data and evidence. 
A solution recommended was basically tying the two terms to conclusions, that is, asking learners to “interpret their 
data to generate evidence to support a conclusion” (p. 14; emphases supplied). The combination of Comparing with 
this aspect (SDE) by 38% of the teachers could be understood in terms, for instance, of this SPS being fundamental 
in differentiating between “collected information” (data) and “what is already known” (see Appendix A). 

Where the teachers were able to select two expected SPS, a fair percentage (i.e., 55% and 49%) of responses 
were found in these aspects: Scientific investigations all begin with a question but do not necessarily test a hypoth- 
esis (SIQ) and Explanations are developed from a combination of collected data and what is already known (EDK). 
Tomkins and Tunnicliffe’s (2001) research eloquently shows that observations form the basis for raising questions 
about phenomena, and formulating explicit hypotheses when teaching and investigations commence. Thus, it 
is reasonable that 55% of the teachers thought Observing melds with SIQ. For them, the phenomenon observed 
was life in the mangroves ecosystem and the subsequent questions raised concerned its adaptation for survival 
(Appendix A). On the other hand, 49% of the teachers paired Accessing & recalling information with EDK. Their prior 
knowledge (see Appendix A) could have been the springboard for their ability to interpret what was observed (Cf. 
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Wiyanto et al., 2022), that is, they accessed and recalled information to explain what the results of the investigation 
meant prior making conclusions (Cf. DBE, 201 1b). 

Very low, yet correct percentage responses were also observed. For instance, 34% of the teachers thought 
Measuring blends with All scientists performing the same procedures may not get the same results (SIR). They 
selected this SPS most often that the rest. This is reasonable because “virtually all scientific observations involve 
counts, measurements, or both” and replicability of scientific results is rooted in measurement precision, methods 
and steps taken (National Academies of Sciences, Engineering, and Medicine [NASEM], 2019, p. 46). The selection of 
the SPS could also be understood through the lens of the South African science curriculum, which defines this SPS 
as “using measuring instruments...” (DBE, 2011b, p. 11). It was interesting that Communicating - an SPS that fits in 
all the aspects of scientific inquiry - was ignored by the teachers. It was only in relation to RCD (i.e., Research conclu- 
sions must be consistent with the data collected) where it was selected more often (26%) than in other aspects. The 
recognition of Communicating in RCD is reasonable because it entails making information known to others through 
textual and pictorial forms, and research conclusions require that learners communicate their findings consistent 
with recorded observations (DBE, 2011b). 


Conclusions and Implications 


The research aimed to draw from pre-service teachers’ fieldwork-based experiences to establish their views 
concerning Scils, scientific inquiry, SDGs and SPS. The research indicates that the teachers had confidence in plan- 
ning and carrying out Scils. It highlighted the importance of using the constructivist model to plan science, engag- 
ing learner interest in investigating in science and discerning and controlling variables in a science investigation. 

Classifying and identifying and exploring were the common types of scientific investigations selected con- 
cerning the mangroves ecosystem studied. The selection of exploring shows that the teachers had challenges in 
identifying the scientific investigation they utilised. However, they succeeded in showing the connections between 
the studied mangroves and SDGs13 and 14. The results further highlighted the importance of observing, access- 
ing & recalling information, measuring and communicating when executing some aspects of scientific inquiry. 

It was concluded that, in teacher education, constructivism, learner engagement and understanding of the 
nitty-gritties of Scils such as manipulation of variables are all important factors in a quest to introducing them (Scils) 
in an ecology-based course. The study further highlights the need to leverage on natural ecosystems to strategically 
position and promote understanding of SDGs and SPS in terms of their connectedness to Scils and scientific inquiry. 

The present research provides small but significant ideas on how to disrupt the higher education science 
curriculum with regard to teacher education by tapping into pre-service teachers’ views concerning Scils’ role in 
advocating ecosystem services restorations and environmental sustainability amid present challenges related to 
our natural ecosystems. It emphasises a need for further research on the connection between SPS and the differ- 
ent aspects of scientific inquiry. Such SPS may include Communicating, which is fundamental to the reporting of 
the nitty-gritty details of a scientific investigation before, during and after its execution. As referred to elsewhere, 
the study has a primary limitation that could also be addressed in future research: Our choice of a sample might 
have weakened external validity because the present teachers’ questionnaire was part of their fieldwork exercise. 

The research implies that science teacher educators that offer courses that entail fieldwork component framed 
around SPS and realisation and enactment of SDGs should design scientific activities and scientific investigations 
tailored to the constructivist model and the tenets of scientific inquiry. The challenges associated with identifying 
the scientific investigation utilised and reconciling SPS, such as Communicating, with different aspects of the scientific 
inquiry point to the need to tap into teachers’ conceptual understanding of these concepts (highlighted). 
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Xiaoqing Xu, 
Jijian Lu 


Introduction 
From MOOCs to SPOCs 


The concept of Massive Open Online Course (MOOC) was proposed by 
Dave Cormier and Bryan Alexander in 2008, and the tsunami of MOOC started 
to sweep across the globe in the late 2012 (Qin &Tan, 2020). In 2013, Tsinghua 
University in China was the first to conduct research on MOOC courses on the 
edX platform, but during the rapid development of MOOC, some problems 
also emerged, such as frustrating completion rates (Jordan, 2015; Yuan & Pow- 
ell, 2013), high dropout (Zhang et al., 2019), limited student-faculty interaction 
(Baggaley, 2013; Wise & Cui, 2018), inappropriate cultural background and 
educational methods (Kang & He, 2018), endangered intellectual property 
(Berrett, 2013), and deficient pedagogical principles (Qin & Tan, 2020). 

In response to these drawbacks of MOOC, Fox proposed to combine 
MOOC course resources with classroom teaching to form a Small Private On- 
line Course (SPOC) in 2013 (Hashmi, 2013). The researcher believes that MOOC 
can be used as a supplementary teaching resource, but it cannot completely 
replace conventional classroom teaching. When it can really improve students’ 
learning efficiency and help teachers impart knowledge, it is a SPOC course 
(Fox, 2013). SPOC courses are different from MOOC courses and conventional 
courses, which are a blend of conventional offline teaching and online learn- 
ing, and a hybrid learning model (Sun, 2019). Online courses mainly refer to the 
use of MOOC lecture videos, whereas offline teaching refers to the teachers’ 
discussion and research of the issues that arise with students to consolidate 
and implement knowledge. 

In 2013, Harvard University offered three SPOC courses, one of which, 
“Copyright Law’, was offered on the edX platform, in the form of online learn- 
ing and offline group discussions, followed by a final exam and a certificate 
for those who passed. This model has been well received and will continue 
to be delivered in the future. Additionally, Fox at the University of California, 
Berkeley, offered a SPOC course on “Software Engineering” through the edX 
platform, which added an automatic scoring function that allowed for timely 
feedback on online learning (Fox et al., 2014). Meanwhile, Fox also promoted 
this model, the hybrid teaching model of online learning and offline classroom 
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integration has achieved excellent teaching results. In contrast to conventional course, SPOC course has its own 
unique style that is more flexible and incorporates more information technology (Coughlan, 2013). 

SPOC courses have also been incorporated into college courses in China. Hua Sun uses a hybrid teaching 
strategy that combines “Rain Classroom” with SPOC to increase teaching efficiency and level while also deepening 
students’ understanding of course content and forming positive teacher-student interactions (Sun, 2019). In ad- 
dition, Fox's research revealed that 10% of the users of MOOC courses are teachers, and they integrate the useful 
parts of the course into the physical classroom through the development and reuse of MOOC video resources (Fox 
et al., 2014), which is not only the development and use of resources for SPOC courses but also the promotion of 
the implementation of SPOC courses. 

The innovative combination of SPOC online learning and offline classroom learning has realized the compre- 
hensive “deep integration” of information technology and education, which transforms the conventional teaching 
model of “acquiring knowledge in class and asking questions after class”to “learning both in the classroom and online 
and online learning assists offline teaching” (Jiang, 2022). The transition from MOOC to SPOC entails a change from 
online courses to hybrid courses, from shallow learning to deep learning, and from various formats to help students 
get a deeper comprehension of knowledge and improve their problem-solving abilities. There are also views that, 
SPOC = Classroom + MOOC (Admiraal et al., 2015; Guo, 2017). 


WeChat Platform 


With the development of information technology, new education models such as flipped classroom, online 
courses and smart classroom have risen rapidly, impacting the original teaching practice system. The WeChat 
platform can carry diversified teaching information, use fragmented times, and go deeper into students life reality, 
so it can be utilized as a platform to combine offline classroom and online learning. The blended learning model 
of the WeChat platform- assisted classroom teaching is also an innovative form of the “Internet+” teaching model. 

WeChat Official Account (WOA) is the most frequently visited platform among college students in China (Hou 
et al., 2017). Functions such as pushing messages and independent interactive communication can be realized on 
WOA.WOA is an Internet-based product that has web end, mobile phone end and PC end. Users only need to make 
a simple registration to apply for WOA. Technically, users can follow the WOA account by scanning the QR code of 
the account, and the administrator of WOA can reply to new followers according to the setting. Teachers can publish 
new graphic messages through the backend and can also analyze students’ reading and learning data. 

There are several advantages in applying WOA to education and teaching. Firstly, WeChat has a huge user 
base, every Chinese college student has a smartphone and a WeChat account and is familiar with using WeChat 
(Zhang, 2016). WeChat has become one of the emerging social software. Secondly, WeChat has lower operational 
cost and is easier to combine with Internet technology compared to other teaching software, without developing 
mobile software, without programming technical foundation, simply by completing the registration of WOA. Thirdly, 
WeChat is instantaneous, with real-time communication functions and social networking features that allow users 
to interact with others (Fan et al., 2015). Instant interaction of messages can be achieved by simply installing the 
WeChat APP and having a mobile network, and managers can also categorize users through the backend, regardless 
of time and space. Therefore, mobile and micro learning is highly possible for its extensive coverage (Liu & Zhang, 
2022; Liu & Jiang, 2015). The fourth is the diversity of information, which can be sent by video, photos, and voice in 
addition to text to completely push graphic messages. The multiple advantages of WeChat make it have the potential 
to become a learning medium. 

Thus, WOA has won widespread acceptance in China for its free, flexible, interactive and more participatory 
method to teaching (Tu et al., 2018), but its function of combining with classroom teaching is limited, the functions 
needed for actual teaching activities are to be further developed. Therefore, other functions need to be added on 
this basis to build a more comprehensive functional WeChat-assisted teaching platform. 


Blended Learning 
Blended learning is now considered to be the most effective and popular teaching model adopted by the educa- 
tion institutions (Rasheed et al., 2020) and has received widespread attention from schools and institutions around 


the world for several reasons. This idea of combining teaching materials with online interventions in an educational 
setting has proven to be an upgrade from the conventional face-to-face and fully online teaching models, bridging 
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the gaps that exist in both of them (Broadbent, 2017; Rasheed et al., 2020). The combination of face-to-face and 
technology-mediated teaching can lead to shorter distances between teachers and students or between students 
and students in online transactions, enhancing the interaction between teachers and students (Kamaruzaman & 
Rouhollah, 2009; Porter et al., 2014; Vaughan, 2007). In addition, compared to face-to-face and fully online teaching, 
blended learning is more preferred by students due to its flexibility in learning time and space, the breadth and 
accessibility of learning resources, and its high level of learning autonomy (Ashton & Elliott, 2007; Collopy & Arnold, 
2009; Poon, 2012; Reiss & Steffens, 2010). 

Several studies have been conducted to compare the effectiveness of blended learning with face-to-face or 
online learning. Academic success and blended learning are positively correlated, according to several research in 
higher education (Allen et al., 2007; Bazelais & Doleck, 2018; Bernard et al., 2014; Deschacht & Goeman, 2015; L6pez- 
Pérezet al., 2011). Blended learning transforms conventional“Duck-stuffing” exam-based education (passive learning 
that usually involves only acquiring knowledge) into active learning where students actively construct knowledge 
(Suartama et al., 2019), resulting in a higher level of learning experience and motivation. 

In addition, numerous research studies have demonstrated that students have greater satisfaction in blended 
learning environments, compared to conventional face-to-face and fully online teaching (Castle & Mcguire, 2010; 
Farley et al., 2011; Martinez-Caro & Campuzano-Bolarin, 2011; Woltering et al., 2009). Blended learning in China has 
been helpful in solving authentic problems in colleges and vocational education, contributed favorably to students’ 
practical training and significantly increased students’ learning satisfaction (Wang et al., 2021). It is predicted that by 
the end of this century, 80-90% of college and enterprise training courses will adopt blended learning (Kyong-Jee 
et al., 2005), and that more than 1 billion learners worldwide will improve their skills in this way (Kim et al., 2009). 

The classroom teaching sessions in the blended teaching model are more flexible, but it is mostly used in higher 
education, where students’ logical thinking skills are better than the secondary education level, and the effect is 
more obvious with the blended teaching model. 


Research Problem 


The WOA platform enables teachers to give students access to top-notch learning resources, monitor their 
progress, and, as needed, deliver individualized training, all of which increases teacher-student interaction as well as 
the internalization and depth of knowledge (Wei, 2017; Yan, 2015). Thus, the impact of WeChat as a social software 
on learning in education has received attention and needs to be explored further (Hou et al., 2021). 

However, most of the empirical studies conducted with WOA are biased towards higher education. WOA is 
mostly used as a supplementary platform for college classroom teaching because it can help students to preview 
before class, expand their knowledge after class, and enhance their academic success (Shen, 2020). Compared how 
well students performed and their ability to use higher-order thinking skills, such as critical thinking and problem- 
solving skills, in the conventional teacher-centered educational model and the WeChat-based flipped classroom 
teaching approach, scholars found that students’ performance and higher-order thinking skills in WeChat-based 
flipped classroom outperformed their peers taught in conventional classrooms(Liu & Zhang, 2022). Whether the WOA 
platform can also be used as a supplementary platform for secondary education is a question worthy of consideration. 

Additionally, previous studies on blended learning have paid attention to secondary education, which have 
found that the commognition of the teacher and the student in a series of computer supported one-to-one online 
tutoring courses experienced three stages: teacher-led commognition process, comparison of commognitive pro- 
cess and student-led commognition process (Lu et al., 2019). Correspondingly, the commognitive responsibility 
gradually shifted from the teacher to the student (Lu et al., 2020), and the proportion of student-led collaborative 
problem solving increased (Lu et al., 2022). Such a blended course based on Constructivism makes students become 
increasingly agile in communication and cognitive interaction, and their cognitive level and class participation are 
gradually improved, which serves to cultivate critical thinking skills. 

Through searching the Chinese Full-text Journal Database with “WeChat Official Account” as the keyword, it 
was found that the research on WeChat platform-assisted classroom teaching model mainly focused on two aspects: 
college classroom and theoretical research. Besides, there is less research on SPOC courses using WeChat platform, so 
the teaching function of the WeChat platform was further developed, and a blended teaching course was designed 
through the combination of SPOC courses and WeChat platform, to study the design and implementation of WeChat 
platform-assisted lower-secondary school science classroom teaching based on SPOC. 

The hypothesis of this study was that integrating a blended teaching model with a lower-secondary school 
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science curriculum in a SPOC environment based on the WeChat platform, results in students exhibiting superior 
performance. Three questions were addressed as follows: 

1. Compared with conventional teaching, whether the model of SPOC-based WeChat-assisted teaching is 
more conducive to improving students’ academic performance, and whether the improvement range 
of students at different levels is consistent? 

2. Compared with conventional teaching, whether the model of SPOC-based WeChat-assisted teaching is 
more conducive to enhancing students’ classroom participation and learning motivation? 

3. Whether the model of SPOC-based WeChat-assisted teaching is effective and applicable in lower- 
secondary school science teaching? 


Research Methodology 
General Background 


This study mainly adopted the action research method. This study constructed a new blended learning model 
based on literature, that is, the SPOC-based WeChat-assisted teaching model. In response to the three research 
questions raised above, this study implemented four cases for teaching practice intervention in two eighth-grade 
classes in the first semester of the 2021 academic year. This study describes the specific process by taking the “Shui 
De Zu Cheng (Composition of Water)" as an example. Qualitative and quantitative methods were used to analyze 
the experimental data. SPSS software was used to conduct quantitative analysis on WeChat backend data, students’ 
academic achievements, classroom participation and other data. Additionally, seven in-service science teachers 
were interviewed to qualitatively evaluate the model. Finally, through the comprehensive analysis of qualitative 
and quantitative data, consider whether the three research questions have received positive responses. 

The research design and process of this study are shown in Figure 1. 


Figure 1 
Research Design and Process 
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Participants 


Participants were from two eighth grade classes of a lower-secondary school in Hangzhou, China, Class 7 
(25 students) and Class 8 (37 students), with different academic performance. None of the participants had any 
previous experience in SPOC-related learning activities. These two classes are randomly selected from the eighth 
grade of the lower-secondary school. Table 1 displays the final examination results of the two classes in the last 
semester. The average scores of the two classes differed by about 38 points, with Class 8 performing better aca- 
demically than Class 7, while the standard deviation of the two classes was not much different, indicating that the 
score distribution of the two classes was relatively uniform. 


Table 1 
Academic Performance before the Implementation of the Teaching Model 


AVE sD SEM 
Class 7 57.92 22.73 4.55 
Class 8 95.59 22.05 3.62 


The results of the independent samples T-test of the final exam scores of the two classes were conducted by 
SPSS, and the results of the test are shown in Table 2. The data in the table shows that the significance (two-sided) 
value is less than .001, and there is a significant difference in the achievement of two classes, so the two levels of 
classes are chosen as experimental classes for the study. 


Table 2 
Difference Test between Final Exam Results of Class 7 and Class 8 (Independent Sample T-test) 


Levene’s equal variance test t-test for equality of mean values 
F p df p (two-sided) MD SE 
paopt enue! 01 92 60 p <.001 37.67 577 
Final exam results valence 
of the last semester : 
Baual varie 50.58 p<.001 37.67 5.81 


is not used 


Construction of SPOC-based WeChat-assisted Teaching Platform 
Application for WeChat Official Account 


As a platform to support classroom teaching, the “I want to learn science” WeChat Official Account was first 
applied for to implement the teaching practice. Students can follow the WOA by scanning the WeChat QR code 
of “I want to learn science” To register for the WOA, you only need to log in to the official registration page of the 
WOA to apply for a subscription number (individuals can only apply for a subscription number), which can be used 
after the auditing is completed. 


Function development of WOA 
The basic functions of WOA can realize the delivery of graphic messages and videos but it cannot provide 


other functions required for classroom teaching. Thus, this study developed WOA twice to build a fully functional 
platform suitable for classroom teaching. 
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(1) Basic Functions 

The basic functions of the WOA include publishing graphic messages, uploading teaching PPTs and videos 
to the material management in the backend, and then publishing the resources through new graphic messages. 
After teachers create a new graphic message in the backend and send it to the group, students can immediately 
receive the message and prepare to learn. 

(2) Expanding Functions 

The functions of WOA are limited and insufficient to fulfill all the requirements for supporting classroom 
teaching, including online test and message board. Therefore, the functions are developed with the help of third- 
party development software. Due to the complicated operation of other development backend software, this 
study chose the relatively simple development software to realize the functions of online test, column diversifica- 
tion and message board. The functions of the WeChat platform are all set up to serve classroom teaching. After 
building the WOA according to different functions, the specific resources are uploaded by different teachers of 
different courses in accordance with their own classes. Teachers only need to upload the resources after setting up 
this WeChat platform, without programming. For teachers, the operation is simple and will not take up too much 
teaching time, and they can check the students’ task completion results through the backend so that teachers can 
keep abreast of students’ knowledge mastery. 

After the function development through the association with the third-party platform, the teaching functions 
that can be realized by WOA can be roughly divided into four parts: resource acquisition (such as teaching PPTs, 
videos, supplementary exercises, knowledge expansion), student display (such as student work display, student 
voting), communication evaluation (such as message board, exchange Q&A), and online test (including pre-class 
preview test and post-class consolidation test). 


Research Procedure 


The teaching design of SPOC-based WeChat-assisted teaching mainly includes three parts: before class, dur- 
ing class and after class (Figure 2). The teacher releases resources through WOA before class, and students receive 
information to preview in advance. During class, the teacher focuses on solving critical problems that exist in 
students’ online preview, and after class, the teacher publishes knowledge expansion on the WeChat platform to 
broaden students’ horizons, and they can also consolidate and enhance students’ knowledge mastery through 
online test function of WeChat platform. 

The students are the main body in this whole process, and every step is based on students’ needs. The powerful 
function of the WeChat platform supports students’ independent learning while facilitating meaningful learning 
to occur. As students learn, they can also forward and share with other students, discuss on the WeChat platform, 
and teachers can participate in the process by giving guidance to students and responding to their questions. In 
this process, the teacher and students form a community through the WeChat platform and classroom teaching. 


Figure 2 
SPOC-based WeChat-assisted Teaching Model 
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a) Knowledge transmission before class. The teacher releases relevant learning resources through the 
WeChat platform before class, students then receive information through smartphone WeChat software 
and study under the supervision of their parents. During the learning process, students can pause for 
reflection at any time, and can also ask questions on the platform at any time to communicate with their 
classmates and teachers. Teachers even can publish resources at different levels for different types of 
students, which requires teachers to develop teaching resources carefully before releasing them, either 
by recording videos of important and difficult points, or by recombining online resources, mainly based 
on students’ needs for the secondary development of resources. Teachers can also issue pre-class task 
lists to help students explore knowledge with tasks. 

b) Knowledge internalization in class. Through the knowledge transmission before class, students already 
have a certain knowledge base. The in-class knowledge internalization session is mainly based on 
classroom teaching, and the content is the most important part of knowledge acquisition. The task list 
before class is also a preparation for the in-class teaching. By focusing on students’ learning dynamics 
through pre-class backend analysis, teachers can analyze the common problems of students and based 
on this, teachers can rearrange classroom activities to solve students’ problems, and improve students’ 
classroom concentration and motivation. 

c) Knowledge solidification after class. After class, the teacher can release the exercises on the WeChat 
platform, and while students answer the questions online, the teacher can understand the difficult 
problems encountered by students in the exercises through the backend. For timely consolidation of 
knowledge, teachers and students can also communicate through the WeChat platform. At the same 
time, teachers can improve the teaching process through students’ online learning performance and 
evaluate students in various aspects. 


A Practical Research in Science of Lower-Secondary School 


The model of SPOC-based WeChat-assisted teaching constructed in the research is divided into three parts: 
knowledge transmission before class, knowledge internalization in class and knowledge solidification after class. 
The following is an example of the case “Shui De Zu Cheng (Composition of Water)’, and the implementation of 
each part is introduced in detail. 


Knowledge transmission before class 


The pre-class preparation of “Shui De Zu Cheng (Composition of Water)” was mainly divided into two parts, 
the first part was the preparation and development of resources, and the second part was the editing and up- 
loading of resources. The development of pre-class resources includes videos, pictures, and audio, etc. For “Shui 
De Zu Cheng (Composition of Water)’, the pre-class resources were mainly videos and task lists. The task list is for 
students to learn and explore knowledge purposefully, but since the task list is text in Word format, it cannot be 
directly posted on the WOA, so it must be converted into pictures first and then uploaded to the backend material 
management for publishing graphic messages. Preliminary processing is also needed for video uploading. Note 
that videos larger than 20M need to be uploaded to Tencent Video for audit first or processed through software, 
and after processing, they also need to be uploaded to the backend material management of WOA first and the 
graphic messages can be released only passed Tencent’s audit. 

The operation video about the electrolysis experiment of water was released because the main points of this 
lesson are the electrolysis experiment of water and the decomposition reaction of water (composition of water). Re- 
leasing the experiment video before class is the basis for the teacher's demonstration experiment in class. Of course, 
the content of the video is not necessarily experimental video but can also be an explanation or an extension of the 
knowledge. Therefore, in addition to uploading the experimental videos, two short videos based on knowledge points 
were also uploaded, reviewing three variations in the physical properties and chemical properties of water, so that 
the students’ memory of their knowledge reserve could be awakened for better teaching during classroom teaching. 

The videos, task lists and other materials were edited and uploaded to the backend of the WOA, the teacher 
consolidated all the resources and released them for all the students of Class 7 and Class 8 to receive the resources 
for learning in the WeChat. After learning, students could interact through the WeChat platform so that teachers 
can comprehend the problems students encountered before class and focus on solving them in class. 
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Knowledge internalization in class 


The in-class session is mainly focused on classroom teaching, with emphasis on the combination of learn- 
ing resources released online. By transferring knowledge before the class, the knowledge transmission in the 
class can be more fluent, the key points can be more prominent, and the depth of knowledge can be deepened 
in the class. 

In the classroom teaching of “Shui De Zu Cheng (Composition of Water)", the demonstration experiment of 
water electrolysis, the generalization for the experimental phenomenon and the extension of the experimental 
conclusion were focused on. Guided by the experiment video and task list before class, students could easily 
conclude that the electrolysis of water gives oxygen at the positive pole, hydrogen at the negative pole, and 
oxygen and hydrogen are produced under the condition of electricity, so the classroom teaching could focus 
on the extension of the experimental conclusion. 

Because the composition of water molecules is very abstract, the composition of water molecules was also 
the focus of classroom teaching, from the experimental conclusion that water is composed of hydrogen (element) 
and oxygen (element), extended to the abstract conclusion that in chemical reactions, molecules can be divided 
into atoms, while atoms can no longer be divided. Additionally, the discussion of students’ questions before class 
revealed that the most difficult point of knowledge for students was the composition of water molecules, from 
macroscopic to the microscopic level. Therefore, in the classroom lecture, microscopic animation was integrated 
into the teaching activities, turning microscopic into intuitive, solving the students’ understanding difficulties 
with intuitive feeling and internalizing knowledge. 


Knowledge solidification after class 


The after-class session of the “Shui De Zu Cheng (Composition of Water)” was divided into three parts. The 
first part was the preparation of PPT resources, each PPT must also be processed into pictures first before being 
uploaded to the material management of the WOA. Through the new graphic message to publish PPT-related 
information, the PPT used in the class will be uploaded to the WOA following the lesson, allowing students to 
supplement the notes not recorded in time during class. For students to find this information quickly, the “PPT” 
secondary menu was set up under the primary menu “Micro. Resources’, students can click the “PPT” menu but- 
ton to jump to receive this graphic message. For previous learning resources, the keyword reply matching was 
set up, when responding to keywords, the corresponding learning resources will pop up, of course, students 
can also check the history of the message to find the previous resources. 

The second part was the after-class online test. Online tests not only provide students with a method to 
consolidate their knowledge but also give teachers a platform to understand the status of students’ homework in 
real time. The function of online test is implemented in the third-party backend, a“Micro. Test” menu was set up 
in the WOA, each after-class practices are linked here, students can click on this menu to receive the practices that 
pop up. The time limit for completing exercises can be set. Each practice question is input in the backend with 
the correct option hidden so that the platform can score and evaluate immediately after the student completes 
the online test. Teachers can check the completion of students’ online tests in the backend and make statistics. 
Additionally, teachers can also note students’ names and categorize them according to their academic status. 

By setting the keyword “Calculate”, students can access the answer page by replying to the keyword “Cal- 
culate” When designing the questions on the editing page, teachers need to input each option and the score 
for each question separately. The question types can be single-choice and multiple-choice, and the rest of the 
questions can be sent to the WOA in the form of pictures. Once the questions are uploaded, they can be posted. 
The interface is simple, and students do not need to switch pages, which is more in line with the cognitive 
characteristics of students. 

Online tests can be set not only after class but also released before class. Following the preview, students 
are given pre-class tests, allowing teachers to easily comprehend the problems that students encountered before 
class and focus on solving them in class to improve the efficiency of teaching. Online tests lighten the teacher's 
burden and maximize class time. 
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The third part is the exchange and Q&A. On the one hand, due to the fact that students have limited time 
to study in the school and need to study many subjects, so they may not always have time to communicate 
face-to-face with teachers in the school, therefore the WOA provides an opportunity to communicate with 
teachers on a one-to-one basis. A message board was set up on the WOA, where students can leave messages 
to their teachers. On the other hand, students can ask questions directly on the platform when they encounter 
problems in online learning or have questions about the classroom learning, the teacher can reply directly in 
the backend. A “Micro. Communication” menu and two sub-menus under it - Question Exchange and Message 
Board- were also set up. By clicking on the message board, students can enter the message board by replying 
“| want to leave a message” according to the prompt. 

Students can discuss questions here, expressing each person's insights and opinions, turning it into a learn- 
ing discussion area where they can engage in interesting informal learning after school. Of course, one-on-one 
communication between students and teachers is also possible to answer questions directly on the WOA. 

Adopting online, offline, and online approaches for the pre-class, in-class, and post-class sessions respectively 
is a type of blended teaching. Blended teaching refers to a multifaceted hybrid teaching model, the essence of 
which refers to the interconnection between online and offline, in-class and out-of-class and interdisciplinary, it 
is a kind of teaching that combines online teaching with conventional teaching, which is defined as “online and 
offline” (Wu, 2022). After-class knowledge solidification is very important, German psychologist Ebbinghaus put 
forward the concept of a forgetting curve, human forgetting for new things is gradual, and the forgetting speed 
is the fastest at the beginning of learning. Therefore, using WOA to assist students in consolidating knowledge 
and improving the efficiency of knowledge mastery when learning new knowledge. 


Data Analysis 


Using a combination of quantitative and qualitative research methods, students’ acceptance of the 
blended teaching model and their mastery of knowledge can be understood, through quantitative analysis of 
WeChat backend data, student academic performance, and their changes in classroom performance behavior. 
Data analysis was conducted by SPSS software to evaluate the design and implementation of the SPOC-based 
WeChat-assisted teaching based on the results of correlation tests between students’ classroom performance and 
academic performance. Interviews were also conducted with seven in-service science teachers to qualitatively 
evaluate and analyze the teaching model from the perspective of in-service teachers, assess the feasibility of 
the model, and make suggestions for subsequent reforms. 

Through a comprehensive analysis, the main teaching design model is proposed to realize the integrated 
application of this blended teaching in a course that develops students’ independent learning skills. 


Research Results 
Based on WeChat Backend Data Analysis 


(1) The function of data statistics in the backend of WOA was used to count the backend click data of “Shui 
De Zu Cheng (Composition of Water)” learning resources, as shown in Figure 3. 
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Statistics of Menu Clicks and Resource Calls in the Backend of WOA 
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Learning resources including PPTs, experimental videos, and online tests were released on September 29 
(at the black arrow) and sent to 62 students and 7 teachers of Class 7 and Class 8. The figure shows that the peak 
period of students’ clicks of learning resources was ushered in at the beginning of the holiday on September 29 
and September 30, and in the days when the holiday was about to end on October 6, 7 and 8. The total number 
of resource calls was 143, indicating that students had repeated learning on the WeChat platform, which provides 
students with a platform and personalized learning space that can be repeatedly learned anytime, anywhere. In 
comparison, the frequency of using the WeChat platform increased significantly after the “I want to learn science” 
WOA was put into use on September 29, indicating that the The WeChat platform can enhance students’ motiva- 
tion and interest in learning. 

(2) The data analysis function of the WOA backend was used to count the click volume data of the hier- 
archical menu of “I want to learn science” WOA, as shown in Figure 3. 

It can be seen that the first-level menu “Micro. Test” received the most clicks, with 121 clicks during this 
period, and 1.51 clicks per capita, demonstrating that students attach great importance to online tests. The num- 
ber of online tests far exceeds the number of students in the class, but each account can only take one online test 
within the specified time, so it is certain that some students shared the learning content with their friends or other 
learners, and other learners who did not follow the WOA also studied and tested by clicking the shared information. 
This shows that the sharing function of WeChat promotes the shareability of resources and expands the audience 
of resources, fully reflecting the feature that Internet learning is not limited by time and space. Secondly, “Micro. 
Resources’, the secondary menu of which had a high click rate of calculation exercises, learning videos and PPT 
menus, with the number of clicks per capita reaching more than 1.00, because these resources can help students 
to learn outside of class. 


Analysis of Academic Performance 


The performance of two classes at two levels before and after the implementation of the model was compared, 
which showed that after the implementation of the two cases through SPOC-based WeChat-assisted teaching, 
students actively conducted online tests on the WeChat platform to consolidate knowledge in a timely manner. 
Subsequently, the following tests of these contents were conducted in two classes, and the statistics of the two 
tests are shown in Figure 4. As can be seen from the figure, the overall distribution of scores after the implementa- 
tion of these two cases is above the final exam results. This indicates that after the online learning sessions before 
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and after class, the students’academic performance improved to a certain extent, especially for the students in the 
lower range. It may be that the students were more interested in the content of the class after the pre-class learn- 
ing, and could concentrate more on the class, so the class was more attractive to students, which can effectively 


improve their learning efficiency. 


Figure 4 


Test Results before and after the Implementation of SPOC-based WeChat-assisted Teaching Model 
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In addition, the test scores of Class 7 and Class 8 were statistically and analytically compared by class (Table 
3), it was found that the average scores of both tests in both classes increased compared to the final scores, and 
the standard deviation decreased. This indicates that the scores of both classes have improved and the dispersion 


of the scores has decreased. 


Table 3 
Analysis of Academic Performance and Classroom Participation 


Class 7 Class 8 
AVE 57.92 95.59 
Final exam results of the last semester SD 22.73 22.05 
SEM 4.55 3.62 
AVE 59.24 99.27 
Shui De Zu Cheng (Composition of Water) SD 21.95 20.22 
SEM 4.39 3.32 
AVE 61.92 100.43 
Wu Zhi De Rong Jie (Dissolution of Sub- SD 24.45 2061 
stances) 
SEM 4.29 3.39 
Before implementation 11 21 
Classroom participation Shui De Zu Cheng (Composition of Water) 26 28 
Wu Zhi De Rong Jie (Dissolution of Substances) 34 31 
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Analysis of Classroom Participation 


As indicated in Table 3, the number of classroom participation of students was also counted in two different 
levels of classes before and after the implementation of the SPOC-based WeChat-assisted teaching model. The 
comparison reveals that after the pre-class knowledge transmission session, the students’ enthusiasm for partici- 
pating in class greatly rose, especially for Class 7, which had a lower overall level, and the students’ class partici- 
pation increased rapidly. Through the pre-class knowledge transmission, students studied the teaching content 
in advance and were able to respond positively to the teacher's questions in class, while the teacher established 
more connections with the pre-class knowledge in class to form knowledge transmission and promote students’ 
understanding of knowledge. 


The Relationship between Academic Performances and Classroom Participation 


The number of classroom participants in the two classes was counted before and after the implementation of 
the two cases. The amount of change in the number of classroom participation was obtained after averaging the 
number of classroom participation in the two classes before and after implementing the cases respectively, and the 
correlation between the test scores and the number of classroom participation was analyzed using SPSS software. 


Table 4 
Correlation Analysis between the Change of Academic Performance and the Change of Students’ Participation in Class 7 


Change value 
of test result of 
“Composition of 


Changes in the 
number of students 
participating in the 


Change value of test 
result of “Dissolution 


i of substances” 
water class 
PCCs 1 .99** AT* 
Change value of test result of , 
“Composition of water” p (two sided) p <.0001 016 
N 25 25 25 
PCCs .99** 1 A6* 
Change value of test result of “Dis- : 
solution of substances” pling sen) pan ne 
N 25 25 25 
PCCs AT* A6* 1 
Changes in the number of students p (two sided) 016 020 
participating in the class 
N 25 25 25 


** Correlation is significant at the .01 level. 
*, Correlation is significant at the .05 level. 


As can be seen from Table 4, the Pearson correlation coefficient between the test scores of “Shui De Zu Cheng 
(Composition of Water)” and the amount of change in the number of times classroom participation is .47, with 
a two-tailed uncorrelated test value of .016, thus rejecting the hypothesis that they are not correlated. The Pear- 
son correlation coefficient between the test scores of “Wu Zhi De Rong Jie (Dissolution of Substances)” and the 
amount of change in the number of times classroom participation is .46, with a significance (two-sided) test value 
of .020. From these two data, a significant correlation is obtained at both the .01 and .05 levels. In other words, the 
more students participate in class, the higher their test scores will be. However, it remains to be further examined 
whether the improvement of students’academic performance is all due to the result of implementing SPOC-based 
WeChat-assisted teaching, but it is certain that the implementation of SPOC-based WeChat-assisted teaching helps 
students’ classroom participation and their mastery of what they have learned. 
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Evaluation and Analysis of In-service Teachers 


During the implementation of the two cases, seven in-service science teachers of the same grade were invited 
to be interviewed afterwards. The WeChat platform was deemed a good platform for supporting the classroom by 
all seven of the teachers. The integration of the core thinking of the SPSS curriculum with the WeChat platform is 
a good teaching method to use in the classroom. There are a variety of online learning courses that are emerging 
and are of great interest to students. For the teacher, the pre-class knowledge transfer session is the first prepa- 
ration for the textbook content, and the implementation of the in-class knowledge internalization session after 
understanding the learning situation is another preparation. After the second preparation, classroom teaching was 
more targeted, and students’ interest in learning and classroom attraction was enhanced. 

However, two teachers noted that the preparation of pre-class resources and post-class online tests would 
take up extra time for teachers, because teachers in lower-secondary school are extremely busy, especially head 
teachers. On the other side, however, the online tests and micro-communication sections of the WeChat platform 
reduced teachers’ time for correcting assignments and communication. Overall, it did not occupy too much for 
teachers. Furthermore, two teachers suggested that the application of this teaching method could be more flexible 
and not all contents are suitable for after-class or before-class sessions. In fact, not all content has to follow this 
model, the WeChat platform is just an auxiliary platform. What materials need to be put on the WeChat platform 
to assist in teaching are determined by the teaching design of the teacher in charge of the class. It just serves as a 
tool to supplement classroom teaching, and this model is flexible and changeable. 

Several other teachers proposed that this is a new teaching method that supplements conventional teaching 
and is full of attraction and novelty. After the pre-class session, the classroom lessons were also more targeted. 
Especially for new teachers, this model is equivalent to secondary preparation. Following the pre-lesson session, the 
students’ learning situation is initially understood, and the secondary preparation improves classroom efficiency, 
the post-class session promotes the consolidation of students’ knowledge while also serving as a time for teachers 
to reflect on teaching, which is conducive to the growth of new teachers. 

In addition, teachers also suggested that more functions could continue to be developed in depth on the 
WOA, such as the student display module. In some highly interactive content, students can be invited to prepare 
the display content, giving them an opportunity to demonstrate their value, and also reflecting the main role of 
students. Make teaching activities more flexible and let students become the masters of the classroom. 


Discussion 


In this study, SPOC-based WeChat-assisted teaching was proposed as a new blended teaching model with 
the aim of studying the role of this teaching model in lower-secondary school science courses. After four cases of 
blended teaching using this model, all three research questions posed in this study were positively responded to. 

Regarding question 1, the changes in academic performance before and after the implementation of the SPOC- 
based WeChat-assisted teaching model in these two classes were compared, and it was found that the academic 
performance of both classes was higher than the final exam results of the previous semester, indicating that this 
blended teaching model has an important role in promoting students’ academic performance. In addition, from 
the analysis of the improvement range of academic performance, it can be concluded that this blended teaching 
model is more effective in promoting the academic development of students at the lower level of the class. 

The literature shows that MOOC provides a platform for students to learn independently and breaks the limita- 
tion of teaching time and space. However, it is open to students on a national or even global scale and the teaching 
principles are defective (Qin &Tan, 2020). It is suggested that students should learn in the SPOC environment rather 
than in the MOOC environment (Fox, 2013). SPOC teaching can deal with high-quality teaching resources differently 
according to the differences of student bases, so as to meet the needs of different students’ learning objectives and 
provide different levels of learning content (Chen, 2022; Qiu, 2022). This blended teaching model provides more 
personalized choices for students to learn specific courses and teaching methods are more targeted. However, Fox 
et al’s research on SPOC still focuses on higher education. This study not only combines the WeChat platform with 
SPOC but also emphasizes the applicability and effectiveness of SPOC in secondary education. 

The results of the study were similar to those of Shen, indicating that the introduction of social media-WeChat- 
as an auxiliary tool to classroom teaching can achieve student-centered teaching and improve students’ ability 
to actively learn (Shen, 2020). However, the study only demonstrated that students who actively participated in 
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WeChat interactions exhibited higher final grades. In contrast, this study compared the positive effects of the 
blended teaching model in all three aspects: longitudinal time level, cross-sectional student base level, and the 
relationship between classroom participation and academic performance. It pointed out that the SPOC-based 
WeChat-assisted teaching model could improve students’ academic performance, and the higher the classroom 
participation the better the academic performance, and that the improvement in academic performance was dif- 
ferent for different levels of students. 

Regarding question 2, by analyzing the changes in students’ classroom participation in both classes before 
and after receiving the blended teaching model, this study shows that the SPOC-based WeChat-assisted classroom 
teaching model can promote students’ classroom participation and enhance their learning motivation. In addition, 
the WeChat backend data reveals that the frequency of students using the WeChat platform to assist their learning 
increased, and more importantly, students used the sharing function of WeChat to share their learning content 
with other classes, all of which indicate that the WeChat platform can enhance students’ motivation to learn and 
improve their learning enthusiasm. 

As proved in the literature, students were inattentive to what was being said in the lecture for 40% of the 
time (Meyers & Jones, 2010). However, interactive teaching involving high-level interaction increases students’ 
involvement that students are devoted to learning to facilitate their engagement (Paul et al., 2014). Under the 
blended teaching model, students can enhance the transformation and absorption of knowledge through more 
exchanges and discussions with teachers and classmates, and effectively improve students’ enthusiasm for learning 
(McLaughlin et al., 2014). Different from other studies on MOOC and blended learning, the SPOC-based WeChat- 
assisted classroom teaching model designed in this study is proposed on the basis of the secondary development 
of WOA in conjunction with third-party platforms, which can better meet the actual teaching needs. 

Sun described a blended teaching model, which combines the “Rain Classroom” and SPOC, forming a good 
atmosphere for interaction between teachers and students, and deepening students’ understanding of knowl- 
edge (Sun, 2019). However, the “Rain Classroom” platform used in this study is a teaching platform that is not used 
frequently. Students need to register using WeChat and become familiar with the operation of this platform for a 
period of time. Different from the “Rain Classroom” used by Sun, this research directly uses the WeChat platform. 
As the social platform with the highest average daily visits in China, almost all Chinese people have a WeChat ac- 
count and can skillfully use it, which avoids the disadvantages of students’ registration process and unfamiliarity 
with the platform and reduces the technical difficulty and time cost. The SPOC-based WeChat-assisted classroom 
teaching model makes up for the high dropout rate of MOOC (Zhang et al., 2019) and the limited interaction 
between teachers and students (Baggaley, 2013; Wise & Cui, 2018). 

Regarding question 3, based on the previous analysis of students’ academic performance, classroom partici- 
pation and WeChat backend data, it is confirmed that the model is applicable and effective to a certain extent for 
lower-secondary school students. Through the analysis of the interview content of seven in-service science teachers, 
from the perspective of in-service teachers, it also reflects the tendency to maintain a positive and supportive at- 
titude towards the blended teaching model and points out that after using the teaching model to prepare lessons 
for the second time and track students’ learning situation, teachers can have a clearer grasp of learning situation 
and teaching can be more targeted. Of course, this model can also be further improved and developed. 

In addition, another important finding of this study is similar to the results of Tong et al. With the help of the 
WeChat platform, the frequency and complexity of students’ classroom participation have increased (Tong et al., 
2022). However, this study also analyzes the correlation between students’ test scores and classroom participation 
and finds that there is a positive correlation between them. The more students participate in the class, the higher 
their test scores will be. 


Conclusions and Implications 


This research combined SPOC and WeChat platform to develop a new blended teaching model. Firstly, the 
development trend of SPOC, the blended teaching model and the application of the WeChat platform in teaching 
were analyzed. Secondly, the SPOC based WeChat-assisted classroom teaching model was developed, and the 
blended teaching process of science in lower-secondary school based on this model was introduced. According 
to the feedback results of qualitative and quantitative data, the following conclusions were drawn: 

(1) The SPOC-based WeChat-assisted classroom teaching model proposed in this study is applicable and 
effective in lower-secondary school science curriculum. 
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(2) This teaching model divides the teaching process into pre-class, in-class and post-class stages, so that 
teaching is no longer limited to the classroom, and the sustainability of teaching and the enthusiasm of students 
for learning have been improved. 

(3) This research combines SPOC and WeChat platform, innovates the classroom teaching model, and covers 
the deficiencies of conventional teaching. Online teaching and offline teaching are organized at the same time to 
make the course more reasonable, facilitate the change of course teaching methods, and help students become 
the main body of classroom teaching. Under this teaching mode, the feedback of learning resources and various 
links is improved, and excellent teaching results have been achieved in lower-secondary school science teaching. 

Based on these, several suggestions and improvement strategies for the use of this model have been proposed 
for reference of front-line teachers only: 

(1) Interface optimization. Lower-secondary school students lack focus on learning. The choice of fonts, pictures 
and materials in the learning interface should be moderate. Optimizing the WeChat learning interface and improving 
the attractiveness of materials can stimulate students’ desire to continue learning and ensure high-quality learning. 

(2) Perfect communication system. A more perfect communication system can be further developed and 
improved to achieve a faster and more effective circulation of learning resources among WeChat friends. 

(3) Build a “fragmented” learning environment. Make full use of the “fragmented” learning advantages of 
WeChat to build a learning space suitable for the WeChat platform, while paying attention to the coherence and 
systematicness of disciplines. 

(4) Establish a mechanism combining multiple evaluation methods. Under this blended teaching model, 
multiple evaluation can be realized, including timely evaluation of teachers, automatic scoring of tests, students’ 
learning performance indirectly reflected by WeChat backend data, and mutual evaluation between students. 

(5) Personalization and diversity need to be integrated. Students with different foundations can obtain learn- 
ing resources that meet their personalized needs on the learning platform to enhance the pertinence of teaching. 
In addition, teachers can make full use of the advantages of the platform to release diversified learning resources 
to stimulate students’ learning motivation. 

The innovation of this study is mainly summarized in the following two aspects. First of all, for the research 
audience, this study transferred the SPOC teaching model to secondary education, expanding the research object of 
the SPOC teaching model; additionally, in terms of platform development, this research chose the WeChat platform 
as the online platform for blended teaching. This platform is widely accepted in China because of its interactive, 
attractive, adaptable, portable, sustainable, and more participatory teaching methods. And combined with the 
characteristics of SPOC courses, the function of the WeChat platform has been re-developed in combination with 
the third-party platform, so that it can meet more teaching needs. 


Limitations and Suggestions for Further Research 


However, there are still some limitations in the current study. Firstly, this research is in line with the new con- 
cept of the current teaching reform, but due to irresistible reality reasons, this research only involves 62 students 
from two classes of Grade 8 in a lower-secondary school in Hangzhou, China. There is no large-scale analysis of 
this blended teaching method in a wider scope. The problems and suggestions identified are only applicable to 
this group of participants. Therefore, further research will be carried out later, including more lower-secondary 
schools, learning stages and participants, to draw general conclusions on common problems. Secondly, this study 
focuses on the applicability of mobile learning methods, mainly using SPSS software for data analysis, and does not 
involve other scientific measurement tools. The use of a more standardized international scale to evaluate the role 
of this teaching model in other aspects is a problem that needs to be explored in the future. Finally, the research 
on the SPOC based WeChat assisted classroom teaching model has just started, and the model proposed in this 
study is just a rudiment, with the hope that it can be further improved through follow-up research, continuous 
practice, and gradually establish relevant theoretical and practical guidance to provide a theoretical basis for the 
implementation and application of WeChat and blended learning in basic education. 


Declaration of Interest 


The authors declare no competing interest. 


https://doi.org/10.33225/jbse/23.22.701 LQ 


rats: 


Journal of Baltic Science Education, Vol. 22, No. 4, 2023 


BLENDED LEARNING ON WECHAT PLATFORM-BASED SPOC IN LOWER-SECONDARY SCHOOL ISSN 1648-3898 /Print/ 
SCIENCE TEACHING 

(PP.701-718) ISSN 2538-7 138 /oniines 
References 


Admiraal, W., Huisman, B., & Pilli, O. (2015). Assessment in massive open online courses. Electronic Journal of e-Learning, 13, 
207-216. https://files.eric.ed.gov/fulltext/EJ1062116.pdf 

Allen, |. E., Seaman, J., & Garrett, R. (2007). Blending in: The extent and promise of blended education in the United States. 
https://files.eric.ed.gov/fulltext/ED529930.pdf 

Ashton, J., & Elliott, R. (2007). Juggling the balls—Study, work, family and play: student perspectives on 
flexible and blended heutagogy. European Early Childhood Education Research Journal, 15(2), 167-181. 
https://doi.org/10.1080/13502930701321378 

Baggaley, J. (2013). MOOC rampant. Distance Education, 34(3), 368-378. https://doi.org/10.1080/01587919.2013.835768 

Bazelais, P., & Doleck, T. (2018). Blended learning and traditional learning: A comparative study of college mechanics courses. 
Education and Information Technologies, 23(6), 2889-2900. https://doi.org/10.1007/s10639-018-9748-9 

Bernard, R. M., Borokhovski, E., Schmid, R. F., Tamim, R. M., & Abrami, P. C. (2014). A meta-analysis of blended learning and 
technology use in higher education: From the general to the applied. Journal of Computing in Higher Education, 26(1), 
87-122. https://doi.org/10.1007/s12528-013-9077-3 

Berrett, D. (2013). Debate over MOOCs reaches Harvard. The Chronicle of Higher Education. https://www.chronicle.com/ 
article/debate-over-moocs-reaches-harvard/ 

Broadbent, J. (2017). Comparing online and blended learner’s self-regulated learning strategies and academic performance. 
The Internet and Higher Education, 33, 24-32. https://doi.org/10.1016/j.iheduc.2017.01.004 

Castle, S. R., & Mcguire, C. (2010). An analysis of student self-assessment of online, blended, and face-to-face learning 
environments: Implications for sustainable education delivery. International Education Studies, 3(3), 36-40. 
https://doi.org/10.5539/ies.v3n3p36 

Chen, J. (2022). Effectiveness of blended learning to develop learner autonomy in a Chinese university translation course. 
Education and Information Technologies. https://doi.org/10.1007/s10639-022-11125-1 

Collopy, R. M. B., & Arnold, J. M. (2009). To blend or not to blend: Online-only and blended learning environments. Teacher 
Education, 18(2), 85-101. https://www.xueshufan.com/publication/2577508772 

Coughlan, S. (2013). Harvard plans to boldly go with ‘SPOCs’ BBC News. http://www.bbc.co.uk/news/business-24166247 

Deschacht, N., & Goeman, K. (2015). The effect of blended learning on course persistence and performance of adult learners: 
A difference-in-differences analysis. Computers & Education, 87, 83-89. https://doi.org/10.1016/j.compedu.2015.03.020 

Fan, W., Yan, M., Kai, L., & Qiu, B. (2015). A practical study in the flipped classroom based on WeChat under environment of 
mobile learning. Open Education Research, 21(3), 90-97. http://en.cnki.com.cn/Article_en/CJFDTOTAL-JFJJ201503012.htm 

Farley, A., Jain, A., & Thomson, D. (2011). Blended learning in finance: Comparing student perceptions of lectures, tutorials 
and online learning environments across different year levels*. Economic Papers: A journal of applied economics and 
policy, 30(1), 99-108. https://doi.org/10.1111/j.1759-3441.2010.00094.x 

Fox, A. (2013). From MOOCs to SPOCs. Communications of The ACM, 56(12), 38-40. https://doi.org/10.1145/2535918 

Fox, A., Patterson, D. A., Ilson, R., Joseph, S., Walcott-Justice, K., & Williams, R. (2014). Software engineering curriculum 
technology transfer: Lessons learned from MOOCs and SPOCs. Electrical Engineering and Computer Sciences. 
https://www2.eecs.berkeley.edu/Pubs/TechRpts/2014/EECS-2014-17.pdf 

Guo, P. (2017). MOOC and SPOC, which one is better? Eurasia Journal of Mathematics Science and Technology Education, 
13(8), 5961-5967. https://doi.org/10.12973/eurasia.2017.01044a 

Hashmi, A. H. (2013). HarvardX set to launch second “SPOC” (Small, Private, Online Course). Distance Education Report. 
https://www.thecrimson.com/article/2013/9/17/kennedy-school-spoc-edx/ 

Hou, J., Ndasauka, Y., Jiang, Y., Ye, Z., Wang, Y., Yang, L., Li, X., Zhang, Y., Pang, L., Kong, Y., Xu, F., & Zhang, X. (2017). 
Excessive use of WeChat, social interaction and locus of control among college students in China. Plos One, 12(8). 
https://doi.org/10.1371/journal.pone.0183633 

Hou, R., Han, S., Wang, K., & Zhang, C. (2021). To WeChat or to more chat during learning? The relationship between WeChat 
and learning from the perspective of university students. Education and Information Technologies, 26(2), 1813-1832. 
https://doi.org/10.1007/s10639-020-10338-6 

Jiang, L. (2022). Factors influencing EFL teachers’ implementation of SPOC-based blended learning in higher 
vocational colleges in China: A study based on grounded theory. Interactive Learning Environments, 1-20. 
https://doi.org/10.1080/10494820.2022.2100428 

Jordan, K. (2015). MOOC completion rates: The data. MOOC Project, 1-2. http://www.katyjordan.com/MOOCproject.html 

Kamaruzaman, J., & Rouhollah, K. (2009). Preliminary study on the role of social presence in blended learning environment 
in higher education. International Education Studies, 2(4), 79-83. https://files.eric.ed.gov/fulltext/EJ1065767.pdf 

Kang, Z., & He, L. (2018). Construction and practice of SPOC teaching mode based on MOOC. International Journal of 
Emerging Technologies in Learning, 13(12), 35-49. https://doi.org/10.3991/ijet.v13i12.9702 

Kim, K.-J., Bonk, C. J., & Teng, Y.-T. (2009). The present state and future trends of blended learning in workplace learning 
settings across five countries. Asia Pacific Education Review, 10(3), 299-308. https://doi.org/10.1007/s12564-009-9031-2 

Kyong-Jee, Kim, Curtis, J., Bonk, TingTing, & Zeng. (2005). Surveying the future of workplace e-learning: The rise of blending, 
interactivity, and authentic learning. eLearn, 2005(6), 2-2. https://doi.org/10.1145/1073198.1073202 

Liu, D., & Zhang, H. (2022). Improving students’ higher order thinking skills and achievement using WeChat 


716 


ASS https://doi.org/10.33225/jbse/23.22.701 


Journal of Baltic Science Education, Vol. 22, No. 4, 2023 


ISSN 1648-3898 /Print/ BLENDED LEARNING ON WECGHAT PLATFORM-BASED SPOC IN LOWER-SECONDARY SCHOOL 
SCIENCE TEACHING 
ISSN 2538-71 38 /Online/ (PP.701-718) 


based flipped classroom in higher education. Education and Information Technologies, 27(5), 7281-7302. 
https://doi.org/10.1007/s10639-022-10922-y 

Liu, H., & Jiang, X. (2015). A design and teaching practice of college English teaching based on WeChat platform. Foreign 
Language and Literature, 4, 138-143. 

Loépez-Pérez, M. V., Pérez-Lopez, M. C., & Rodriguez-Ariza, L. (2011). Blended learning in higher education: 
Students’ perceptions and their relation to outcomes. Computers & Education, 56(3), 818-826. 
https://doi.org/https://doi.org/10.1016/j.compedu.2010.10.023 

Lu, J., Tao, Y., Xu, J., & Stephens, M. (2020). Commognitive responsibility shift and its visualizing in computer-supported 
one-to-one tutoring. Interactive Learning Environments. https://doi.org/10.1080/10494820.2020.1777167 

Lu, J., Tuo, P., Feng, R., Stephens, M., Zhang, M., & Shen, Z. (2022). Visualizing commognitive responsibility shift in 
collaborative problem-solving during computer-supported one-to-one math tutoring. Frontiers in Psychology. 
https://www.frontiersin.org/articles/10.3389/fpsyg.2022.815625/full 

Lu, J., Zhang, X., & Stephens, M. (2019). Visualizing the commognitive processes in computer-supported one-to-one tutoring. 
Interactive Learning Environments, 27(5-6), 645-654. https://doi.org/10.1080/10494820.2019.1610452 

Martinez-Caro, E., & Campuzano-Bolarin, F. (2011). Factors affecting students’ satisfaction in engineering 
disciplines: traditional vs. blended approaches. European Journal of Engineering Education, 36(5), 473-483. 
https://doi.org/10.1080/03043797.2011.619647 

McLaughlin, J. E., Roth, M.T., Glatt, D. M., Gharkholonarehe, N., Davidson, C. A., Griffin, L. M., Esserman, D. A., & Mumper, R. 
J. (2014). The flipped classroom: A course redesign to foster learning and engagement in a health professions school. 
Academic Medicine, 89(2), 236-243. https://doi.org/10.1097/ACM.0000000000000086 

Meyers, C., & Jones, T. B. (2010). Promoting active learning. Strategies for the college classroom. Biochemistry & Molecular 
Biology Education. https://doi.org/10.1016/0307-4412(94)90198-8 

Paul, Baepler, J., D., Walker, Michelle, & Driessen. (2014). It’s not about seat time: Blending, flipping, and efficiency in active learning 
classrooms - ScienceDirect. Computers & Education, 78(Sep.), 227-236. https://doi.org/10.1016/j.compedu.2014.06.006 

Poon, J. (2012). Use of blended learning to enhance the student learning experience and engagement in property education. 
Property Management, 30(2), 129-156. https://doi.org/10.1108/02637471211213398 

Porter, W. W., Graham, C. R., Spring, K. A., & Welch, K. R. (2014). Blended learning in higher education: Institutional adoption 
and implementation. Computers & Education, 75, 185-195. https://doi.org/10.1016/j.compedu.2014.02.011 

Qin, M. X., & Tan, X. (2020). Examining a SPOC experiment in a foundational course: design, creation and implementation. 
Interactive Learning Environments, 1-18. https://doi.org/10.1080/10494820.2020.1722710 

Qiu, X. (2022). Blended teaching mode of higher vocational english based on MOOC plus SPOC. Wireless Communications 
& Mobile Computing, 2022. https://doi.org/10.1155/2022/9320161 

Rasheed, R. A., Kamsin, A., & Abdullah, N. A. (2020). Challenges in the online component of blended learning: A systematic 
review. Computers & Education, 144. https://doi.org/10.1016/j.compedu.2019.103701 

Reiss, M., & Steffens, D. (2010). Hybrid toolboxes: Conceptual and empirical analysis of blending patterns in application 
of hybrid media. Ukio Technologinis ir Ekonominis Vystymas, 16(2), 305-326. https://doi.org/10.3846/tede.2010.20 

Shen, J. (2020). Introduction of social media to aid active-learning in medical teaching. Interactive Learning Environments. 
https://doi.org/10.1080/10494820.2020.1766508 

Suartama, |. K., Setyosari, P., Sulthoni, & Ulfa, S. (2019). Development of an instructional design model for mobile 
blended learning in higher education. International Journal of Emerging Technologies in Learning, 14(16), 4-22. 
https://doi.org/10.3991/ijet.v14i16.10633 

Sun, H. (2019). A SPOC teaching mode of college english translation based on “Rain Classroom” /nternational Journal of 
Emerging Technologies in Learning, 14(17), 182-193. https://doi.org/10.3991/ijet.v14i17.11206 

Tong, P., Yin, Z., & Tsung, L. (2022). Student engagement and authentic language use on WeChat for learning Chinese as a 
foreign language. Computer Assisted Language Learning. https://doi.org/10.1080/09588221.2022.2052906 

Tu, S., Yan, X., Jie, K., Ying, M., & Huang, C. (2018). WeChat: An applicable and flexible social app software for mobile teaching. 
Biochemistry and Molecular Biology Education, 46(5), 555-560. https://doi.org/10.1002/bmb.21170 

Vaughan, N. (2007). Perspectives on blended learning in higher education. International Journal on E-learning, 6(1), p.81-94. 
https://www.redalyc.org/pdf/706/70643085002.pdf 

Wang, L., Huang, Y., & Omar, M. K. (2021). Analysis of blended learning model application using text mining method. 
International Journal of Emerging Technologies in Learning, 16(1), 172-187. https://doi.org/10.3991/ijet.v16i01.19823 

Wei, Z. F. (2017). Construction of higher vocational english teaching model based on WeChat. Campus English(41), 56-57. 
https://kns.cnki.net/kcms/detail/detail.aspx?FileName=XY YY201741049&DbName=CJFN2017 

Wise, A.F.,& Cui, Y. (2018). Learning communities in the crowd: Characteristics of content related interactions and social relationships 
in MOOC discussion forums. Computers & Education, 122, 221-242. https://doi.org/10.1016/j.compedu.2018.03.021 

Woltering, V., Herrler, A., Spitzer, K., & Spreckelsen, C. (2009). Blended learning positively affects students’ satisfaction and 
the role of the tutor in the problem-based learning process: results of a mixed-method evaluation. Advances in Health 
Sciences Education, 14(5), 725-738. https://doi.org/10.1007/s10459-009-9154-6 

Wu, C. (2022). Effect of online and offline blended teaching of college english based on data mining algorithm. Journal of 
Information & Knowledge Management, 21. https://doi.org/10.1142/S0219649222400238 

Yan, J. F. (2015). A review of the researches on WeChat. International Conference on Social Science and Technology Educantion. 
https://www.docin.com/p-1732142144.html 


ray 


https://doi.org/10.33225/jbse/23.22.701 QT 


Journal of Baltic Science Education, Vol. 22, No. 4, 2023 


BLENDED LEARNING ON WECGHAT PLATFORM-BASED SPOC IN LOWER-SECONDARY SCHOOL ISSN 1648-3898 /Print/ 
SCIENCE TEACHING 
(PP.701-718) ISSN 2538-7138 soniines 


Yuan, L., & Powell, S. (2013). MOOCs and open education: Implications for higher education. https://doi.org/10.13140/2.1.5072.8320 

Zhang, L. (2016). Research on the practice of flipped class mode based on the platform of WeChat. Journal of US-China 
Public Administration, 13(5), 358-366. https://doi.org/10.17265/1548-6591/2016.05.005 

Zhang, X. M., Yu, J. Y., Yang, Y., Feng, C. P., Lyu, J., & Xu, S. L. (2019). A flipped classroom method based on a small private 
online course in physiology. Advances in Physiology Education, 43(3), 345-349. https://doi.org/10.1152/advan.00143.2018 


Received: November 01, 2022 Revised: May 30, 2023 Accepted: June 10, 2023 


Cite as: Xu, T., Xu, J., Xu, X., & Lu, J. (2023). Blended learning on WeChat platform-based SPOC in lower-secondary school science 
teaching. Journal of Baltic Science Education, 22(4), 701-718. https://doi.org/10.33225/jbse/23.22.701 


Tugen Xu PhD, Associate Professor, Institute of Chinese Educational 
Modernization, Jinghengyi School of Education, Hangzhou Normal 
University, Hangzhou, China. 
E-mail: xutugen@163.com 


Jiaying Xu Master, Postgraduate, Jinghengyi School of Education, Hangzhou 
Normal University, Hangzhou, China. 
E-mail: 1354270782@qq.com 


Xiaoging Xu Master, In-service Teacher, Hangzhou Yuhang Xianlin Middle 
School, China. 
E-mail: 1299704256@qq.com 
Jijian Lu PhD, Associate Professor, Institute of Chinese Educational 
(Corresponding author) Modernization, Jinghengyi School of Education, Hangzhou Normal 


University, Hangzhou, China. 
E-mail: lujijian@foxmail.com 
ORCID: https://orcid.org/0000-0003-0208-9486 


718 


UNS https://doi.org/10.33225/jbse/23.22.701 


@® 6) This is an open access article under the 
(c) Creative Commons Attribution 4.0 
BY NC 


International License 


ASSESSMENT OF CONCEPTUAL 
UNDERSTANDING IN STUDENT 
LEARNING OF MOON PHASES 


Yi Zou, Lizhen Jin, 
Xiao Huang, 
Yanbing Li 


Introduction 


Basic scientific concepts are one of the main contents of science edu- 
cation in lower-secondary school, so helping students better understand 
scientific concepts is the key to achieving the corresponding goals of science 
education. Developing students’ scientific literacy has become the fundamen- 
tal goal of science education worldwide in recent years, and helping students 
understand basic scientific concepts is undoubtedly the basis and premise for 
improving their scientific literacy (Murcia, 2009; National Research Council, 
2008; Ozdem et al., 2010). Many studies have shown that students’ ability to 
solve science-related problems in real situations (the main performance of 
scientific literacy) depends on their understanding of corresponding scientific 
concepts (Gerace et al., 2001; Martinez et al., 2021; Nie et al., 2019). Different 
from the direct, concrete and self-evident daily concepts, scientific concepts 
usually have the typical characteristics of logic and abstraction (Nersessian, 
2002; Yun, 2020). For example, Magnetic Induction Line is not an objec- 
tive object, but a virtual curve used to describe and explain some relevant 
magnetic phenomena. This means that students usually need to experience 
a thinking process of metaphor and modeling when understanding scien- 
tific concepts such as Magnetic Induction Line (Barsalou, 1991; Nersessian, 
2002). Therefore, it is of great significance to assess students’ conceptual 
understanding in learning specific scientific concepts because it can reflect 
and diagnose students’ cognitive characteristics, especially the cognitive 
difficulties, so as to provide evidence for targeted educational interventions 
(Dai et al., 2019; Xu et al., 2020). 

The concept topic of this study is Moon Phases, a typical scientific 
concept that requires modeling process for accurate understanding. After 
reviewing previous related literature, it was found that students have many 
difficulties in learning Moon Phases, especially in understanding the nature 
of Moon Phases, that is, establishing the periodic correspondence among 
the observed phenomena of moon phases, the relative spatial positions and 
optical effects of the three celestial bodies (the moon, the earth, and the 
sun), and the perspectives of human beings on the earth (Chastenay, 2016; 
Mulholland & Ginns, 2008). However, existing related research studies have 
deficiencies in the evaluation and analysis of different levels of understanding 
of students in the process of learning this nature, which restrict teachers to 
take corresponding effective educational interventions to help them. This 
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Abstract. Many research studies have 
shown that lower-secondary school 
students’ scientific literacy depends on 
their understanding of corresponding 
scientific concepts. Therefore, it is of great 
significance to model and assess students’ 
conceptual understanding in learning 
specific scientific concepts, because it can 
reflect and diagnose students’ cognitive 
characteristics, so as to provide evidence 
for targeted educational interventions. 
However, on the topic of Moon Phases, 
existing research studies have deficiencies 
in the evaluation and analysis of 

different levels of students’ conceptual 
understanding, which restrict teachers to 
take effective educational interventions to 
help them. Thus, this study explored a new 
level division and an assessment test for 
revealing and explaining students’ different 
conceptual understanding in learning 
Moon Phases. This study was conducted 
among 768 Chinese lower-secondary 
school students. Through a variety of 
quantitative and qualitative analyses, 

it was found that students’ conceptual 
understanding of Moon Phases can be 
divided into three levels of continuous 
improvement, including the mechanical 
memory of different phenomena of moon 
phases, the formation of a preliminary 
mental model of Moon Phases, and 

the construction of a complete mental 
model of Moon Phases. Accordingly, this 
study made some precise suggestions for 
effectively improving students’ conceptual 
understanding of Moon Phases. 
Keywords: assessment of conceptual 
understanding, level division, Moon Phases, 
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study is a new exploration in this direction. To make a distinction, the “Moon Phases” with uppercase letters in 
this article refer to the scientific concept, and the “moon phases” with lowercase letters in this article refer to the 
observed phenomena. 


Literature Review (Student Learning of Moon Phases) 


In science education, there is a commonly accepted viewpoint that students tend to construct their personal 
understanding of the things through direct interactions with the world around them (Thouin, 1997). However, such 
understanding style may sometimes lead to students’ misunderstanding of scientific concepts (Chastenay, 2016; 
Driver, 1981; Thouin, 2004). The phases of the moon are natural phenomena that students can often observe, but 
Moon Phases is a relatively difficult scientific concept for them (Chastenay, 2016; Ucar, 2014). Relevant research 
studies have found that students have many mistakes in learning Moon Phases, some of which are caused by 
their daily observation experience, such as the phases of the moon occur due to‘the shadow of the earth; ‘planet 
or the sun casts a shadow’ ‘cloud covers the moon; and so on (Baxter, 1989; Chastenay, 2006; Liu, 2005; Schoon, 
1992; Stahly, 1999). From these mistakes, it can be seen that the relevant daily experience is not an assistant but 
an obstacle for students to understand Moon Phases. This fact is consistent with some explanations of the reasons 
for students’ misconceptions in astronomy learning, that is, human beings’ exclusively geocentric perspective on 
the sky, coupled with human beings’ inability to judge the characteristics of astronomical systems, is the source 
of many corresponding misconceptions (Black, 2005; Gazit et al., 2005; Miller & Brewer, 2010; Nussbaum, 1985; 
Trumper, 2001). There is no exception in the topic of Moon Phases, the misconceptions mentioned above have been 
reported by relevant surveys in the past decades, which exist not only among lower-secondary school students, 
but also among college students, and even among science teachers (Jones & Lynch, 1987; Nielsen & Hoban, 2015; 
Ogan-Bekiroglou, 2007). 

Therefore, it has always been the core strategy of the teaching of Moon Phases to help students understand 
the phases of the moon through constructing corresponding mental models, rather than relying on direct observa- 
tion experience. For this matter, traditional teaching on Moon Phases usually applies physical three-dimensional 
models to demonstrate the relative positions of the relevant celestial bodies (the sun, the moon, and the earth) 
and combines discussion and analysis of the corresponding optical effects and visual effects to help students 
establish the mental model, so that they can explain the moon phase phenomena they observed (Trundle et 
al., 2007; Ucar, 2014). Computer-supported teaching is the other major type of teaching that is used to promote 
students to develop a proper mental model of Moon Phases. The philosophy of this teaching type is similar to 
that of traditional teaching, but the celestial models, the optical effects, and the visual effects are presented in 
some digital forms believed to be more effective in the construction of mental models, such as animation, video 
project, software program, and so on (Barnett & Morran, 2002; Bell & Trundle, 2008; Hobson et al., 2010; Keating 
et al., 2002; Sun et al., 2009). It can be seen that both traditional teaching and computer-supported teaching are 
aimed at leading students to overcome the limitation of the geocentric point of view on the sky, and form an al- 
locentric mental model of the moon phase phenomena that is closer to the scientific view. According to Sadler 
(1992), without establishing a mental model of the periodic correspondence among the observed phenomena of 
moon phases, the relative spatial positions and optical effects of the three celestial bodies, and the perspectives 
of human beings on the earth, students will not understand the nature of Moon Phases and will find many related 
problems virtually impossible to solve (Sadler, 1992). 

Previous research studies mentioned above have provided many insightful conclusions about students learn- 
ing Moon Phases. However, these research studies generally have only made a rough analysis and judgment on 
whether students understand the nature of Moon Phases after receiving the related lessons, but have not made an 
accurate and detailed assessment on the different conceptual understanding that students may have in learning 
Moon Phases (Bell & Trundle, 2008; Chastenay, 2016; Subramaniam & Padalkar, 2009; Ucar, 2014), which makes it 
difficult for teachers to carry out targeted interventions to help students further improve their conceptual under- 
standing of Moon Phases according to their own existing understanding statuses. 


Research Aim 


To solve the deficiencies of the assessment on students’ conceptual understanding of Moon Phases, this study 
explored a new level division to present and describe the development of students’ conceptual understanding of 
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Moon Phases. Meanwhile, based on the division, this study designed and implemented a test among 768 Chinese 
lower-secondary school students to assess and analyze their actual conceptual understanding status in learning 
Moon Phases. In addition, based on the research findings, this study conducted some corresponding suggestions 
and discussions for improving students’ conceptual understanding of Moon Phases. In summary, the three goals 
of this study were: (1) Construct a level division to map out students’ conceptual understanding of Moon Phases. 
(2) Employ an assessment based on the level division to analyze the characteristics of students’ conceptual under- 
standing in learning Moon Phases. (3)Make some suggestions for improving students’ conceptual understanding 
of Moon Phases according to the research findings. 


The Level Division for Mapping out Students’ Conceptual Understanding in Learning Moon Phases 


Although the assessment approaches of students’ understanding of scientific concepts adopted by the ex- 
isting research studies were various, most of them reflected a progressive thinking mode (Dai et al., 2019; Liu et 
al., 2022). In other words, these approaches tended to divide students’ understanding of scientific concepts into 
several different levels from low to high and then developed and used corresponding assessment instruments 
(such as questionnaires, observation scales, test questions, etc) to analyze and present students’ understanding of 
scientific concepts (Dai et al., 2019; Liu et al., 2022; Xie et al., 2021; Xu et al., 2020). Among them, the most classic 
was the’Structure of the Observed Learning Outcome (SOLO)’ taxonomy proposed by Professor Biggs of Australia 
based on Piaget's theory of cognitive development stages. The SOLO theory held that students’ understanding of 
the concept can be divided into five levels from concrete to abstract, from one-dimensional to multi-dimensional, 
from unordered to ordered: pre-structural, uni-structural, multi-structural, relational, and extended abstract (Biggs 
& Collis, 1982; Stalne et al., 2016). Similar to the SOLO theory was the Learning Progressions (LPs) theory that has 
emerged in recent years in US science education reform. LPs theory emphasized that students’ understanding of 
the scientific concept is a gradual development process from one state to another. With the continuous leap of 
understanding state, students will form a better cognitive structure related to the concept, a clearer way of think- 
ing anda stronger ability to solve situational problems (Alonzo & Steedle, 2009; Battista, 2011). In addition, Linn et 
al. expressed the students’ understanding level of the scientific concept as the degree of knowledge integration, 
that is, whether and to what extent meaningful links among the concept and other relevant scientific concepts, 
scientific principles, scientific methods, and scientific thinking have been established in students’ cognitive struc- 
ture (Linn, 2006; Liu et al., 2022). 

On the basis of the above theoretical views and literature review of Moon Phases, 3 professors of science edu- 
cation and 3 outstanding science teachers of Chinese lower-secondary school (all the 3 teachers have taught for 
more than 15 years) were invited to form an expert group to explore the corresponding level division of students’ 
conceptual understanding in learning Moon Phases through consultation and demonstration. The following is the 
detailed process of the level division. 

Consistent with the above-mentioned tendency of students to understand things, the expert group believed 
that students will give priority to their direct observation experience of moon phases when learning the concept 
of Moon Phases. In other words, students’ original understanding of Moon Phases is to regard it as a series of ob- 
served phenomena. Although the teaching of Moon Phases in Chinese lower-secondary school is also committed 
to helping students construct the mental model, teachers usually provide methods of mechanically memorizing 
the periodic phenomena of moon phases for those students who cannot think beyond the observation experi- 
ence. Thus, the expert group determined that the low level of understanding of Moon Phases is that students have 
mechanically remembered the periodic phenomena of moon phases. 

Different from the understanding status of low level students, most of the students will start to construct a 
relevant mental model in learning Moon Phases. According to the expert group, in the process of a complete mental 
model construction, students need to build a preliminary mental model of the periodic relative spatial positions 
of the moon, the earth, and the sun from the cosmic perspective at first (see Figure 1). Meanwhile, students need 
to analyze and master the optical characteristics in this preliminary mental model. As shown in Figure 1, consid- 
ering that the relative position of the earth and the sun changes little within one month, it is possible to focus 
only on the position change of the moon relative to the earth. And considering the great distance between the 
sun and the earth & the moon, it is possible to equate the sunlight with the parallel light. These two components 
are consistent with the treatment in the teaching of Moon Phases in Chinese lower-secondary school. To sum up, 
the expert group determined that the intermediate level of understanding of Moon Phases is that students have 
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constructed a preliminary mental model as shown in Figure 1, which reflects the periodic relative spatial positions 
and optical effects of the three celestial bodies corresponding to the periodic phenomena of moon phases from 


the cosmic perspective. 


Figure 1 
The Intermediate Level of Understanding of Moon Phases 


Sun 


However, the brightness and darkness of the moon in the preliminary mental model are sometimes different 
from the observed phenomena of moon phases on the earth. This is because the various phenomena of Moon 
Phases are a series of visual effects observed by human beings on the earth, while the preliminary mental model 
lacks the perspective of human beings on the earth. Therefore, in order to form a complete mental model of the 
concept of Moon Phases, it is necessary to add the human observation perspectives on the earth and the corre- 
sponding visual effects to the preliminary mental model. Accordingly, the expert group proposed a mental model 
as shown in Figure 2 below to present the understanding of the high level students on the concept of Moon Phases. 
This mental model accurately maps out the periodic correspondence between the observed phenomena of moon 
phases and the relative spatial positions of the three celestial bodies, and the optical effects contained therein. 


Figure 2 
The High Level of Understanding of Moon Phases 
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Research Methodology 
Background 


In the procedure of research, considering the representativeness of the sample (which can largely represent 
the general academic level of Chinese lower-secondary school students), relatively suitable participants were 
selected. Next, the assessment test was designed according to the above-level division of students’ conceptual 
understanding of Moon Phases. Before and after the Moon Phases lesson, all the participants took the pre-test and 
post-test. Then, the overview analysis, the scores change trend analysis, the exploratory factor analysis, and the 
thinking process analysis were carried out to comprehensively and deeply explore the different characteristics of 
the students’ conceptual understanding of Moon Phases. 


Participants 


The participants of this study were from a lower-secondary school in Jinhua City, Zhejiang Province, China. Jinhua 
is a moderately developed city in China, and this lower-secondary school is at the medium quality level in Jinhua, 
so the academic level of the participants largely represents the general academic level of Chinese lower-secondary 
school students. Chinese lower-secondary school students learn an introductory level integrated science course, 
and the Moon Phases is arranged to study in the 7th grade. There were 20 classes in the 7th grade of this school, 
with a total of 768 students. All 768 students participated in this study. This study was approved by the Zhejiang 
Normal University Review Board, and informed consents were obtained for all participants. The average age of these 
students was 12.8 years old, including 390 boys and 378 girls. The academic performance of the 20 classes was 
similar, and the contents and strategies of the Moon Phases lesson adopted by the science teachers of these classes 
were similar. In late May 2022, all the 768 students attended a 45-minute lesson on Moon Phases in their own class. 


Design of the Assessment Test 


Based on the above level division, a test consisting of 12 single choice questions (1/4) was designed to assess 
students’ different understanding in learning Moon Phases. All the 12 questions were adapted from the ones that 
students often encounter in their daily study. For each question, it was tried to avoid the interference of irrelevant 
information. The questions were put into 3 sets according to the varying degrees of understanding required, in- 
cluding simple set, preliminary set, and integrated set. 

The simple set contains 3 questions (Q1, Q6, Q8), which can be solved by mechanically retelling the relevant 
information about the phenomena of moon phases in memory. For example, Q6 gives the time and location of 
a specific phenomenon of the moon phase and asks students to choose the correct picture of this moon phase. 
It was expected that most students can answer these questions correctly after learning the Moon Phases in the 
related lessons, not to mention the teacher will provide some mechanical memory tricks. 

The preliminary set contains 4 questions (Q3, Q7, Q10, Q12), which correspond to the intermediate level of 
understanding of Moon Phases. In other words, students need to have constructed the preliminary mental model 
to correctly answer these questions. It was expected that the students at the high level can answer all of these 
questions correctly, the students at the intermediate level should often be able to answer most of these questions 
successfully, but the students at the low level can’t get the correct answers generally. 

The integrated set contains 5 questions (Q2, Q4, Q5, Q9, Q11), which can be solved by students when they 
have constructed the complete mental model. It was expected that the students at a high level should be able to 
solve most of these questions, the students at an intermediate level should have difficulty with these questions 
but may succeed in some of them, and the students at a low level may not be able to answer any questions cor- 
rectly because they can only guess. 

The detailed test questions are attached in the Appendix section. Each question was evaluated with ‘1’ or ‘0; 
‘1’for the correct answer and ‘0’ for the incorrect answer. Then, the correct rate was used to express the student's 
score. 160 Chinese lower-secondary school students who have learned the Moon Phases lesson were randomly 
selected for a preliminary test. Through analysis, it is found that the Cronbach's alpha of the whole test is .89, and 
the Cronbach's alpha of the three question sets are .87, .85, and .91 respectively, indicating that the test has good 
structural validity. 
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The Moon Phases Lesson Received by the Participants 


The teaching of Moon Phases in Chinese lower secondary is usually conducted in a 45-minute lesson. At the 
beginning of the lesson, the teacher presents the students with different phenomena of moon phases, and asks them 
a question: How do these phenomena of moon phases come into being? After briefly responding and analyzing 
the students’ answers, the teacher guides the students to establish the following hypothesis: different phenomena 
of moon phases may be formed due to the different visual effects of the different relative spatial positions of the 
moon, the earth, and the sun. Then, the teacher leads the students to carry out an exploratory activity as shown 
in Figure 3 below to verify the hypothesis. The detailed operation and analysis of the exploratory activity are as 
follows: (1) assume the sun is on the blackboard, as the huge distance between the sun and the earth & the moon, 
the sunlight can be regarded as the parallel light from the blackboard; (2) take a half blackened ping-pong ball as 
the moon, and make the half that is not blackened (white) always face the sunlight, so as to simulate the brightness 
and darkness of the moon from the cosmic perspective; (3) take the student as the observer on the earth, and let 
him/her hold the ping-pong ball and rotate counterclockwise; (4) let the student record the shape of the white 
part of the ping-pong ball observed at each position from A to H; (5) introduce the name, occurrence time and 
other relevant information of the moon phase observed at positions A to H; (6) guide the students to construct 
the mental model through discussing and analyzing the hypothesis and the exploratory activity. In China, there is 
a special calendar based on the law and period of moon phases change, which is called the lunar calendar, where 
the first day of each month is the new moon, and the fifteenth day of each month is the full moon, and so on. At 
the end of the lesson, the teacher provides a memory formula of the periodic changes of the phenomena of moon 
phases corresponding to the lunar calendar to the students, so that the students can mechanically remember 
them. One part of this memory formula is first first first right right, third second second left left! which means that 
the first quarter moon appears in the first half of the month, in the first half of the night, in the right half of the 
sky, and the right half of the moon is bright; the third quarter moon appears in the second half of the month, in 
the second half of the night, in the left half of the sky, and the left half of the moon is bright. The other part of this 
formula involves the matching information of the remaining specific moon phases such as the new moon, the full 
moon, etc. On the whole, in this lesson, the teacher not only tries to guide the students to construct the mental 
model of Moon Phases through an exploratory activity but also provides the students with the tricks of mechani- 
cally remembering the different phenomena of moon phases. 


Figure 3 
The Activity of Exploring the Causes of Different Phenomena of Moon Phases 
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Data Collection and Analysis 


Before and after the Moon Phases lesson, all the 768 students took the pre-test and post-test. Each test was 
completed in 30 minutes. Additionally, 100 of the 768 students were randomly selected to continue to attend 
think-out-loud interviews. They were required to explicate the thinking process when dealing with questions in 
detail, so that their conceptual understanding could be interpreted more clearly. Each interview lasted for about 
30 minutes and was audiotaped. (According to similar studies conducted in the past, the selection ratio of 100/768 
was sufficient to meet the needs of qualitative analysis. If it failed, more students would be selected for interview 
and analysis.) 

In order to comprehensively and deeply explore the different characteristics of students’ conceptual under- 
standing of Moon Phases, some quantitative and qualitative methods were applied (the quantitative analyses were 
carried out with the aid of Statistical Package for Social Science (SPSS) version 25.0). Specifically, the overview of 
the pre-test and post-test performances of the students were analyzed to measure the improvement of students’ 
conceptual understanding of Moon Phases after receiving the lesson. The scores distribution for different ques- 
tion sets of different overall performances was analyzed to reveal the development characteristics of students’ 
conceptual understanding of Moon Phases. The exploratory factor analysis (EFA) was carried out on the students’ 
post-test data to explore the structural characteristics of students’ conceptual understanding. At the same time, 
by reference to the level division, students’ thinking processes when answering questions were explored through 
analyzing the interview records to further validate the quantitative outcomes. 


Research Results 
The Improvement of Students’ Conceptual Understanding of Moon Phases 
The overview of the pre-test and post-test performances of all the 768 students is given in Figure 4. It can be 


seen that the overall performances of the students have improved significantly[ .39 — .64, t (767) = 40.111, p <.001, 
d=1.447], but the degree of improvement in each question set is different. The improvement of the simple question 


Figure 4 
The Pre-test and Post-test Performances of the Students (with error bars denoting standard error) 
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set[ .49 > .88, t (767) = 35.096, p < .001, d= 1.266] is far superior to that of the preliminary question set[ .42 > .71,t 
(767) = 27.838, p < .001, d= 1.004] and the integrated question set[ .31 — .43, t (767) = 15.050, p < .001, d= 0.543], 
and the improvement of integrated question set is narrowest. These results initially manifest that the level division 
of students’ conceptual understanding of Moon Phases is relatively reasonable. For students, it is the easiest to 
mechanically memorize the phenomena of moon phases, the second is to construct the preliminary mental model, 
and the most difficult is to construct the complete mental model. 


The Structural Characteristics of Students’ Conceptual Understanding of Moon Phases 


The results of exploratory factor analysis (EFA) of the students’ post-test data are plotted in Figure 5, providing 
new detailed information for further analysis of students’ conceptual understanding after the implementation of 
the Moon Phases lesson. As shown in Figure 5(a), there are three factors with eigenvalues greater than 1, which 
explains 53.363% of the cumulative variance (i.e., 20.111%, 19.286%, and 13.966% for factors 1, 2 and 3, respectively). 
And Figure 5(b) provides the factor loading of all the questions on these three factors. It can be seen that the three 
factors exactly correspond to the three question sets, factor 1 represents the simple set, factor 2 represents the 
preliminary set, and factor 3 represents the integrated set. Figure 5(b) also shows that the students’ responses on 
the simple set are moderately correlated to their responses on the preliminary set (.499). However, the students’ 
proficiency in solving the questions of the simple set (.080) and the questions of the preliminary set (.119) is less 
connected to that of the integrated set. These results represent that the students’ conceptual understanding of 
Moon Phases can be divided into three levels as expected, and it may be the most difficult for them to reach the 
high level, that is, to construct a complete mental model of Moon Phases. 


Figure 5 
The EFA Results of all the 768 Students’ Post-test Data 
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The Development Characteristics of Students’ Conceptual Understanding of Moon Phases 


In order to reveal the development characteristics of students’ conceptual understanding of Moon Phases, 
the score distribution for different question sets of different overall performances was analyzed. The students were 
assigned to five groups according to their ranking of total scores in the post-test, each containing 20% of the total 
sample. As shown in Figure 6, the scores on the integrated set are consistently below the scores on the simple set 
and the preliminary set for all groups. The students with low total scores (<20%) can successfully answer most of 
the questions on the simple set, but fail to obtain correct answers on the preliminary set and integrated set. This 
is not different from the expected performances of low level students mentioned above. With the increase of the 
total scores (20%-60%), the performance gaps between the integrated set and the other two sets are more and 
more pronounced, while the performance gap between the simple set and the preliminary set is narrower and 
narrower, indicating that students in this range have begun to establish the preliminary mental model of Moon 
Phases so that they can perform better and better on the preliminary set. With the further improvement of the 
total scores (60%-80%), the performance gaps between the different question sets are less and less pronounced, 
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indicating that students in this range have begun to construct the complete mental model of Moon Phases so that 
they can achieve occasional success in some integrated questions. At last, as for the students with the high total 
scores (Top 20%), there is no significant difference in their scores on different question sets, indicating that these 
students have formed a well-integrated mental model of Moon Phases so that all the questions are within their 
ability. To sum up, these results reveal a general progression of students’ development of conceptual understand- 
ing about Moon Phases, which is very consistent with our preset. 


Figure 6 
The Scores Distribution for Different Question Sets of Different Overall Performances (with error bars denoting standard error) 
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The Structural Characteristics of Conceptual Understanding of the Students with the High Total Scores 


The structural characteristics of conceptual understanding of the students with the high total scores (Top 20%) 
were further explored using the exploratory factor analysis (EFA). As shown in Figure 7(a), the first eigenvalue can 
explain nearly 70.256% of the variance (see Figure 7(b) for the loading of all the questions on the single factor). This 
indicates that these students’ performance can be fully explained by a single salient factor, which means a quite 
integrated conceptual understanding. When compared to the EFA results of all the students, which have three 
distinctive factors, this merging effect further indicates that these students with high total scores have formed the 
complete mental model of Moon Phases. By the way, considering the meaning of EFA analysis of these students’ 
data may be controversial because they answered most of the questions correctly, and the characteristics of their 
conceptual understanding about Moon Phases need to be further interpreted in combination with the subsequent 
qualitative analysis. 


Figure 7 
The EFA Results of the Students with the High Total Scores 
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The Students’ Conceptual Understanding of Moon Phases Reflected by Their Thinking Process 


According to the responses of these 100 students in think-out-loud interviews, their conceptual understand- 
ing of Moon Phases can be divided into three levels as our previous division. 

Low Level: Students at this level can only mechanically remember the different phenomena of moon phases 
without a meaningful reasoning process. It can be said that these students’ conceptual understanding of Moon 
Phases was still at the level of observation experience, and they did not even have an awareness of model con- 
struction. For instance, student A and student B were able to correctly solve most of the simple questions using 
the memorized formula, but they performed poorly on the preliminary questions and the integrated questions. 


StudentA: To be honest, | still don’t understand the cause of the phenomena of moon phases, retelling the memory formula 
is my unique solution strategy. If! can rely on this formula to get the answer, | will choose that answer, otherwise, 
| will have to guess. 

Student B: — Although | know that the phenomena of moon phases are related to the relative spatial positions of the three 
celestial bodies, it is too abstract for me. | really can’t build such a mental model in my mind, so | have no idea 
about how to deal with many questions in this test, except those that just match the formula. 


Intermediate Level: Students at this level have not only established the relationship between the phenomena 
of moon phases and the memory formula but also constructed the preliminary mental model. However, these 
students still cannot effectively construct the complete mental model. Therefore, they answered most simple ques- 
tions and preliminary questions correctly, but they were helpless to solve the integrated questions. The following 
are excerpts from interviews with students who exhibit this level of conceptual understanding of Moon Phases. 


Student C: (For question 7) The sunlight can be regarded as the parallel light, and no matter where the moonis, the half facing the 
sunlight is illuminated. The answer is A. (For question 5) In this question, the right half of the moon is illuminated 
by the sunlight, but there is no such option in the answers. | finally chose C because it is closest to the right half of 
the moon being illuminated. (The correct answer is B). 

Student D: (For question 3) The picture shows that the left half of the moon is illuminated by sunlight, so the sun should be on 
the left side of the moon, and the sunlight can be regarded as the parallel light. The answer is A. (For question 10) 
The moon phase given in the question is not the standard case where half of the moon is illuminated, which is 
too difficult for me. Therefore, | just guessed the answer A. (The correct answer is B). 


High Level: Students at this level have constructed the complete mental model. In other words, in these stu- 
dents’ understanding of the concept of Moon Phases, the phenomena of moon phases, the periodic relative spatial 
positions and optical effects of the three celestial bodies, and the human observation perspective on the earth 
and the corresponding visual effects, are not isolated from each other, but logically connected. Therefore, these 
students were able to correctly solve almost all questions, especially the integrated questions. 


Student E: (For question 4) In this question, although the right half of the moon is illuminated by sunlight when a man is in the 
position shown in the figure, the bright part of the moon he observed should be the left half. The correct answer 
is D. (For question 5)When the moon is in position d, its right half is illuminated by sunlight, but the visual effect 
we see on the earth is that the left side of the moon is bright. In addition, due to the angle, we can only see a small 
part of the moon is bright, as shown in Figure B. So the answer to this question is B. 

Student F: (For question 2) When the moon is in this position, the part that is actually illuminated by sunlight is exactly the right 
half, but when we observe this moon on the earth, the bright part in the visual effect is more than half. It’s C. (For 
question 9) This question is similar to question 2, but the difference is that the bright part of the moon observed 
on the earth is exactly the right half of the moon that is illuminated by sunlight. The answer is B. 


According to the statistics, the number at each level of the 100 interviewed students included 39, 37, and 24, 


respectively. And their average performances on the post-test are shown in Table 1. It can be seen that the results 
in Table 1 are basically consistent with some quantitative analysis results above. 
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Table 1 
The Number and Percentage at Each Level of the 100 Interviewed Students 


Conceptual understanding level Total Simple set Preliminary set Integrated set 
Low (39) 37 68 40 16 
Intermediate (37) 62 87 16 35 
High (24) 97 1.00 1.00 .94 


The Preference for Memory Matching of Some Intermediate and High Level Students 


In addition, this study also found an interesting common feature among the interviewed students with differ- 
ent understanding levels, that is, quite a few of them prefer to solve problems by memory matching. For the low 
level students, this feature is predictable because they only mechanically remember the different phenomena of 
moon phases by reciting the memory formula. Surprisingly, some students at the intermediate and high levels, 
although they have constructed the mental model of Moon Phases to a certain extent, still gave priority to using 
the memory formula when answering questions. The followings are descriptions of the thinking processes of two 
sample students who showed this feature. 


Student G (intermediate level): In answering these questions, my basic strategy is to give priority to using the memory formula 
taught by the teacher. If it fails, | will do other thinking and analysis. 

Student H (high level): Before starting to answer the questions, | wrote the memory formula on the edge of the test paper, and 
also drew the complete mental model of Moon Phases on the edge. In fact, many of the questions in this test were 
answered by matching. When only a few questions can't be directly matched, | would rethink and reanalyze to 
find the answer. 


Discussion 
Moon Phases is Indeed a Difficult Concept to Understand for Students 


Consistent with the findings of previous studies (Chastenay, 2016; Subramaniam & Padalkar, 2009; Ucar, 2014), 
this study also found that Moon Phases are indeed a difficult concept to understand for students. Although stu- 
dents’ performances on the test have improved significantly after receiving the Moon Phases lesson, the degrees 
of improvement are not ideal. The overview of all students’ post-test data shows that the total accuracy of students 
barely exceeds .6. The average score of students in the integrated set is only more than .4, and the average score 
of students in the preliminary set is only more than .7. These results indicate that most of the students have great 
difficulties in constructing the mental model of Moon Phases. The EFA analysis of all students’ post-test data also 
fully verifies this point, which shows that students’ conceptual understanding of Moon Phases presents three levels 
of little relevance, meaning that students’ understanding of Moon Phases has not yet formed a logical integrated 
whole. The results of the qualitative analysis also show that only about 24% of the students may really understand 
the nature of Moon Phases. Put these together, it can be said that students will encounter many difficulties in 
learning Moon Phases, so as the corresponding precise diagnosis and educational interventions are extremely 
necessary for improving students’ understanding of Moon Phases. 


There are Three Levels of Conceptual Understanding in Student Learning of Moon Phases 


On the basis of literature review and expert consultation, this study developed a level division and a correspond- 
ing test to map out and assess students’ different conceptual understanding of Moon Phases. It is found that this 
approach can effectively identify and present students’ understanding of Moon Phases into three different levels, 
including the mechanical memory of different phenomena of moon phases, the formation of a preliminary mental 
model of Moon Phases, and the construction of a complete mental model of Moon Phases. Meanwhile, the analysis 
of scores distribution for different question sets of different overall performances and the EFA analysis of the top 
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20% of students’ data showed that there seems to be a developmental advancement among the three levels of 
conceptual understanding of Moon Phases. In addition, the qualitative analysis found that some intermediate and 
high level students also have a preference for memory matching, which is the only strategy for low level students 
to solve related problems. In view of these findings, it is necessary to carry out different intervention strategies to 
improve different students’ conceptual understanding of Moon Phases. 


Different Corresponding Intervention Strategies Should be Adopted 


To improve the conceptual understanding of Moon Phases of students with different levels, different inter- 
vention strategies should be adopted. For the students at a low level, since they only mechanically remember the 
periodic phenomena of moon phases and rely on scattered memory matching to solve related problems, helping 
them form an awareness of model construction in learning Moon Phases is the first thing the teacher needs to 
strive to do. Specifically, the teacher should remind these students to give up the tendency to learn by rote and 
guide them to try to construct the connection between the phenomena of moon phases and the corresponding 
relative spatial positions and optical effects of the three celestial bodies by looking for certain logical relationships 
and reasoning pathways. Thus, they can solve problems analytically through meaningful reasoning and explana- 
tion, rather than through inefficient mechanical memory. For the students at an intermediate level, they seem to 
be locally connected between the phenomena of moon phases and the preliminary mental model, but they still 
have not constructed the complete mental model of Moon Phases. Therefore, the teacher should try to make these 
students figure out the difference between the preliminary mental model and the complete mental model. The 
core of this difference is that the part of the moon that is actually illuminated by sunlight is not necessarily equal to 
the phenomenon of moon phases observed by humans on the earth, the phenomena of moon phases are visual 
effects based on the preliminary mental model. Thus, these students can effectively solve the related problems 
that need to be solved by applying the complete mental model of Moon Phases. For the students at a high level, 
they seem to have established effective connections among the phenomena of moon phases, the relative spatial 
positions and optical effects of the three celestial bodies, and the perspective of human beings on the earth. This 
means that they have developed a quite deep conceptual understanding of Moon Phases. What the teacher needs 
to do for these students is to remind them to avoid using memory matching to solve problems any more, and 
encourage them to explore more logical relationships and reasoning pathways to link more relevant knowledge 
elements, so as to further expand the formed mental model of Moon Phases and the problem-solving strategies. 
As a result, they may solve more complex and difficult related problems. 


Conclusions and Implications 


Overall, this study shows that the method of constructing the level division and the corresponding assessment 
test is effective in modeling and explaining lower-secondary school students’ different conceptual understanding 
in learning Moon Phases. The results also show that it is necessary to take some precise interventions for effectively 
improving students’ conceptual understanding of Moon Phases accordingly. At the same time, this study can also 
bring some enlightenment to the learning and teaching of scientific concepts in general. Many research studies 
have shown that it is necessary to find an effective way to assess the students’ actual understanding of the specific 
scientific concept and advance it to the nature of this scientific concept. In this regard, this study undoubtedly 
provides a set of strategies for reference both theoretically and practically. This study also has some limitations that 
need to be duly noted. On the one hand, this study was conducted in China, and the adaptability of the research 
findings to a broader context needs to be further tested. On the other hand, this study has fully shown that Moon 
Phases is a very complex and difficult concept for students, so as more diversified assessment approaches need 
to be further explored and verified to investigate students’ conceptual understanding more comprehensively. 
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Appendix 


The Assessment Questions Used in This Study 


1. On the night of March 22 (lunar calendar), what the phenomenon of moon phase can Xiaoming observe in the 
left half of the sky? (D). 
A.New moon B. First quarter moon C. Full moon D. Third quarter moon 


2. As shown in the figure, it is the relative position of the earth and the moon during the moon revolution. When 
the moon moves to position@), what the phenomenon of moon phase can be observed on the earth? (C). 


QO .__ 
ey 
. © Sun 
® 
A.First quarter moon © B.Third quarter moon O 


C.Waxing gibbous moon ®) D.Waning gibbous moon ©) 


3. As shown in the figure, it is the relative position of the earth and the moon. At this time, the left half of the 
moon is illuminated by sunlight in the universe, so the sun is located at (A). 


aS 


Moon Earth 


A. Left side of the Earth & Moon _B. Right side of the Earth & Moon 
C. Upper side of the Earth & Moon _ D. Under side of the Earth & Moon 


4. As shown in the figure, the sunlight shines on the earth and the moon. When a man is in the position shown in 


the figure, what the phenomenon of moon phase can he observe? (D). 
Earth 


Sun 


“ 


HITT 


Moon © 


A. New mnoon B. First quarter moon @) C. Full oe D. Third quarter moon D 
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5. As shown in the figure, it is the orbit of the earth and the moon. When the moon is in position d, the phenom- 


enon of moon phase that can be observed on the earth that night is (B). 
gaeserre=™ ~<----. meee, 


A. Waxing crescent moon ® B. Waning crescent moon oO 


/-_." 
} 


. q) 
) ( 
C. Waxing gibbous moon Q / D. Waning gibbous moon “™ 


6. The Mid-Autumn Festival ( August 15 of the lunar calendar) is a traditional festival in China. Please tell me the 
moon phase you can observe that night is (C). 


A. New mnoon B. First quarter moon C. Full a D. Third quarter noon 


7. The figure shows the relative positions of the sun, the earth and the moon. Which half of the moon in the uni- 
verse will be illuminated by sunlight at this time? (A). 


A. Left O er. D, C. upper @ D. Below e 


8. The Qixi Festival (July 7th of the lunar month) is a traditional festival in China. The correct description of the 
phenomenon of moon phase on the Qixi Festival is (A). 

A. It appears in the right half of the sky in the first half of the night, and the right half of the moon is bright. 

B. It appears in the left half of the sky in the second half of the night, and the left half of the moon is bright. 

C. It appears in the left half of the sky in the first half of the night, and the right half of the moon is bright. 

D. It appears in the right half of the sky in the second half of the night, and the left half of the moon is bright. 


9. As shown in the figure, a is the sun, b is the earth, and c is the moon. When the positions of the sun, the earth 
and the moon are as shown in the figure, which of the following is the closest moon phase that can be observed 
on the earth that night? (B). 


---H--e 


--@---" te, 

s x re. 

4 Y . 

1 ’ 

x b, bs 
taon-7% \ 
a 
‘ , 


A. New noon B. First quarter moon @ C. Full ©. D. Third quarter moon 
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10. As shown in the figure, it is the relative position of the earth and the moon. At this time, the right half of the 
moon is illuminated by sunlight in the universe, so the sun is located at (B). 


© 


Moon 


Earth 


A. Left side of the Earth & Moon _B. Right side of the Earth & Moon 
C. Upper side of the Earth & Moon _ D. Under side of the Earth & Moon 


11. When the moon phase is waning gibbous moon, where is the position of the moon roughly in the figure? (C). 
Earth 


@ Moon 


A. between | and2 B. between 2 and3 C. between 5and6 _D. between 7 and 8 


12. The figure shows the relative positions of the sun, the earth and the moon. Which half of the moon in the uni- 
verse will be illuminated by sunlight at this time? (D). 


A. not be illuminated © B. all are illuminated ©) 


C. below side is illuminated wy, D. right side is illuminated © 
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